{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "id": "huFZurQ5H6Ge",
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "source": [
    "# 第6章 卷积神经网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cdtYxLrmH6Gf"
   },
   "source": [
    "## 6.1 从全连接层到卷积"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fHXEMzORH6Gg"
   },
   "source": [
    "### 练习 6.1.1\n",
    "\n",
    "假设卷积层(6.3)覆盖的局部区域 $\\Delta = 0$。在这种情况下，证明卷积内核为每组通道独立地实现一个全连接层。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QJmj0EV6H6Gg"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xPSRZGQ_H6Gg"
   },
   "source": [
    "&emsp;&emsp;局部区域 $\\Delta=0$ 表示卷积核的大小等于输入的大小。实际就是问，$1×1$ 的卷积核是否等价于全连接（参见本书 $7.3$ 节：`NiN`网络结构）。因此，每个卷积核只能覆盖一个像素点，在这种情况下，卷积层的计算方式与全连接层非常相似。因为每个卷积核只能看到一个通道的信息，相当于每个卷积核只是一个全连接层的权重矩阵。 所以，卷积内核可以看作是每组通道独立地实现一个全连接层。每个卷积核都有自己的权重，每个输入通道都被独立处理，输出通道是各个输入通道的加权和。这种独立处理的方式有效地减少了权重的数量，从而降低了计算成本，并且能够提取出输入数据中的空间特征。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "KDK67aUiH6Gg",
    "outputId": "7a6b52e0-493d-421f-ad12-6258356a3399"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.1741, -0.1579,  0.1226, -0.1115,  0.3360],\n",
      "        [ 0.3191,  0.2607,  0.5800,  0.4441,  0.7841]],\n",
      "       grad_fn=<AddmmBackward0>)\n",
      "tensor([[ 0.1741, -0.1579,  0.1226, -0.1115,  0.3360],\n",
      "        [ 0.3191,  0.2607,  0.5800,  0.4441,  0.7841]],\n",
      "       grad_fn=<ReshapeAliasBackward0>)\n"
     ]
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch库\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "\n",
    "# 定义一个神经网络类MyNet1，继承自nn.Module\n",
    "class MyNet1(nn.Module):\n",
    "    def __init__(self, linear1, linear2):\n",
    "        super(MyNet1, self).__init__()\n",
    "        # 初始化两个线性层\n",
    "        self.linear1 = linear1\n",
    "        self.linear2 = linear2\n",
    "\n",
    "    def forward(self, X):\n",
    "        # 定义前向传播过程：先展平输入X，然后通过两个线性层\n",
    "        return self.linear2(self.linear1(nn.Flatten()(X)))\n",
    "\n",
    "# 定义另一个神经网络类MyNet2，也继承自nn.Module\n",
    "class MyNet2(nn.Module):\n",
    "    def __init__(self, linear, conv2d):\n",
    "        super(MyNet2, self).__init__()\n",
    "        # 初始化一个线性层和一个卷积层\n",
    "        self.linear = linear\n",
    "        self.conv2d = conv2d\n",
    "\n",
    "    def forward(self, X):\n",
    "        # 定义前向传播过程\n",
    "        X = self.linear(nn.Flatten()(X))  # 先展平输入X，然后通过线性层\n",
    "        X = X.reshape(X.shape[0], -1, 1, 1)  # 调整X的形状以适应卷积层的输入\n",
    "        X = nn.Flatten()(self.conv2d(X))  # 通过卷积层，然后再次展平\n",
    "        return X\n",
    "\n",
    "# 初始化线性层和卷积层\n",
    "linear1 = nn.Linear(15, 10)  # 创建一个线性层，输入特征数为15，输出特征数为10\n",
    "linear2 = nn.Linear(10, 5)   # 创建另一个线性层，输入特征数为10，输出特征数为5\n",
    "conv2d = nn.Conv2d(10, 5, 1)  # 创建一个卷积层，输入通道数为10，输出通道数为5，卷积核大小为1x1\n",
    "\n",
    "# 将卷积层的权重和偏置参数重塑并分别赋值给linear2的权重和偏置\n",
    "linear2.weight = nn.Parameter(conv2d.weight.reshape(linear2.weight.shape))  # 重塑卷积层的权重并赋值给linear2的权重\n",
    "linear2.bias = nn.Parameter(conv2d.bias)  # 赋值卷积层的偏置参数给linear2的偏置参数\n",
    "\n",
    "# 实例化两个网络\n",
    "net1 = MyNet1(linear1, linear2)  # 创建net1，将linear1和linear2作为参数传递\n",
    "net2 = MyNet2(linear1, conv2d)  # 创建net2，将linear1和conv2d作为参数传递\n",
    "\n",
    "# 创建一个随机输入张量X，形状为(2, 3, 5)\n",
    "X = torch.randn(2, 3, 5)\n",
    "\n",
    "# 打印两个网络对同一输入X的输出结果\n",
    "# 两个结果实际存在一定的误差，直接print(net1(X) == net2(X))得到的结果不全是True\n",
    "print(net1(X))  # 使用net1进行前向传播并打印结果\n",
    "print(net2(X))  # 使用net2进行前向传播并打印结果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BX-MU4UWH6Gi"
   },
   "source": [
    "### 练习 6.1.2\n",
    "\n",
    "为什么平移不变性可能也不是好主意呢？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GjZs5ROiH6Gi"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nVHNkPmfH6Gi"
   },
   "source": [
    "&emsp;&emsp;平移不变性是一种在信号处理和图像分析中常见的特性，尤其是在卷积神经网络（`CNNs`）中。它意味着一个系统或函数对输入数据的平移是不变的，即如果输入数据发生平移，输出不会改变。这对于很多应用来说是有益的，比如在图像识别中，无论物体在图像中的位置如何，模型都能识别它。\\\n",
    "&emsp;&emsp;平移不变性可能也存在一些局限性或不足之处:\\\n",
    "&emsp;&emsp;1.**丢失空间信息**：当模型对位置不敏感时，它可能无法识别对象的确切位置或对象之间的空间关系。在某些任务中，如场景理解或对象定位，这种空间信息非常重要。\\\n",
    "&emsp;&emsp;2.**不适用于所有任务**：对于一些特定的任务，如图像中文本的识别或布局分析，平移不变性可能不是一个理想的特性，因为这些任务需要对位置和排列非常敏感。\\\n",
    "&emsp;&emsp;3.**过度泛化**：平移不变性可能导致模型过度泛化，无法识别某些应该被视为不同的模式或对象。例如，在医学成像分析中，肿瘤的确切位置对于诊断至关重要。\\\n",
    "&emsp;&emsp;4.**缩放和旋转问题**：虽然平移不变性处理位置的变化，但它不处理缩放或旋转，这可能是图像识别中的关键因素。\\\n",
    "&emsp;&emsp;5.**计算效率**：为了实现平移不变性，卷积网络通常需要更多的参数和计算资源，这可能导致效率低下，尤其是在资源受限的环境中。\\\n",
    "&emsp;&emsp;6.**局部性限制**：平移不变性通常通过局部感受野实现，这可能限制模型捕捉长距离依赖或大尺度结构的能力。\n",
    "\n",
    "&emsp;&emsp;参考：[https://arxiv.org/pdf/1805.12177.pdf](https://arxiv.org/pdf/1805.12177.pdf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AwOkiEoTH6Gj"
   },
   "source": [
    "### 练习 6.1.3\n",
    "\n",
    "当从图像边界像素获取隐藏表示时，我们需要思考哪些问题？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m-VNPxDKH6Gj"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n5z7Gh2xH6Gj"
   },
   "source": [
    "&emsp;&emsp;从图像边界像素获取隐藏表示时，需要考虑一些特定的问题和挑战：\\\n",
    "&emsp;&emsp;1. **边界效应**：在图像边界处，像素的上下文信息可能不完整，这可能导致在提取特征时出现边界效应。边界像素没有足够的邻近像素来形成完整的局部模式，这可能影响隐藏表示的质量。\\\n",
    "&emsp;&emsp;2. **信息丢失**：边界像素通常不如图像中心区域的像素具有丰富的信息。在处理边界像素时，可能会丢失对整体图像理解至关重要的上下文信息。\\\n",
    "&emsp;&emsp;3. **尺度和旋转问题**：在边界区域，图像的尺度和旋转变化可能对隐藏表示产生更大的影响，尤其是在处理尺寸不一或旋转图像时。\\\n",
    "&emsp;&emsp;4. **填充策略的选择**：在使用卷积神经网络等工具时，可能需要对边界进行填充（如零填充）以保持特征图的尺寸。不同的填充策略可能会对边界像素的隐藏表示产生不同的影响。\\\n",
    "&emsp;&emsp;5. **噪声和伪影**：边界区域可能更容易受到图像处理过程中引入的噪声和伪影的影响，这可能会干扰隐藏表示的准确性。\\\n",
    "&emsp;&emsp;6. **特定应用的需求**：根据应用的不同，边界像素的处理方式可能需要调整。例如，在某些任务中，边界信息可能尤为重要，而在其他任务中，则可能不那么重要。\\\n",
    "&emsp;&emsp;为了有效地从图像边界像素获取隐藏表示，可能需要采用特殊的技术和策略，如使用特殊的卷积核、采用适应性边界处理方法或通过数据增强来模拟边界效应。此外，了解和评估边界像素对最终任务的影响也是很重要的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jmJosLlWH6Gj"
   },
   "source": [
    "### 练习 6.1.4\n",
    "\n",
    "描述一个类似的音频卷积层的架构。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NCd3SLSwH6Gj"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zktnQ2nWH6Gj"
   },
   "source": [
    "&emsp;&emsp;一种基于卷积神经网络的音频特征生成方法，首先对声音信号进行预处理和离散傅里叶变换计算声音信号的幅度谱，形成二维谱图信号；然后搭建以上述二维谱图信号为输入的一维卷积神经网络并进行模型训练，得到特征生成器模型；最后对待测声音进行预处理和离散傅里叶变换得到二维谱图信号，并将其送入训练好的一维卷积神经网络，通过卷积网络计算，得到输出即为所要生成的音频特征，实现声音信号的音频特征生成。\n",
    "\n",
    "&emsp;&emsp;参考：https://blog.csdn.net/david_tym/article/details/112756785"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IGBTjTXVH6Gj"
   },
   "source": [
    "### 练习 6.1.5\n",
    "\n",
    "卷积层也适合于文本数据吗？为什么？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xGi_RvNkH6Gk"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "il8LGBpTH6Gk"
   },
   "source": [
    "&emsp;&emsp;卷积层也适合于文本数据。\\\n",
    "&emsp;&emsp;在自然语言处理中，文本数据通常表示为词向量矩阵，其中每行代表一个词的向量表示。卷积层可以在这个矩阵上进行卷积操作，类似于图像卷积层中对图像进行卷积操作。 在卷积层中，卷积核会在输入矩阵上进行滑动窗口计算，输出一个新的特征矩阵。在文本数据中，这个特征矩阵可以看作是对输入文本的不同`n-gram`特征的提取。例如，一个大小为 $3$ 的卷积核可以提取出输入文本中每个长度为 $3$ 的`n-gram`特征。这些特征可以用于后续的分类或者回归任务。 此外，卷积层还可以与循环神经网络（`RNN`）结合使用，形成卷积神经网络（`CNN`）和循环神经网络（`RNN`）的混合模型。这种模型可以同时捕捉文本中的局部特征和全局特征，提高模型的性能。 因此，卷积层适用于文本数据，可以对文本数据进行卷积操作，提取出不同`n-gram`特征，并且可以与`RNN`结合使用，提高模型的性能。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f8asaRppH6Gk"
   },
   "source": [
    "### 练习 6.1.6\n",
    "\n",
    "证明在式(6.6)中，$f * g = g * f$。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QeBwPGyRH6Gk"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yZOvmmOkH6Gk"
   },
   "source": [
    "&emsp;&emsp;通过式(6.6)的定义，我们可以得到：\n",
    "\n",
    "$$(f * g)(x) = \\int_{-\\infty}^{\\infty}f(y)g(x-y)dy$$\n",
    "\n",
    "$$(g * f)(x) = \\int_{-\\infty}^{\\infty}g(y)f(x-y)dy$$\n",
    "\n",
    "&emsp;&emsp;要证明$f * g = g * f$，即证明：\n",
    "\n",
    "$$\\int_{-\\infty}^{\\infty}f(y)g(x-y)dy = \\int_{-\\infty}^{\\infty}g(y)f(x-y)dy$$\n",
    "\n",
    "&emsp;&emsp;为了证明上式成立，我们将其中一个积分的变量名改为$t=x-y$，则有：\n",
    "\n",
    "$$\\int_{-\\infty}^{\\infty}f(y)g(x-y)dy = \\int_{-\\infty}^{\\infty}f(x-t)g(t)dt$$\n",
    "\n",
    "&emsp;&emsp;再将这个式子代回式(6.6)中：\n",
    "\n",
    "$$(f * g)(x) = \\int_{-\\infty}^{\\infty}f(x-t)g(t)dt$$\n",
    "\n",
    "&emsp;&emsp;对比式(6.6)和上面的式子，可以发现它们的形式是完全一样的，只是积分变量名不同而已。因此，我们可以得到：\n",
    "\n",
    "$$(f * g)(x) = \\int_{-\\infty}^{\\infty}f(y)g(x-y)dy = \\int_{-\\infty}^{\\infty}g(y)f(x-y)dy = (g * f)(x)$$\n",
    "\n",
    "&emsp;&emsp;因此，$f * g = g * f$，证毕。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JnoQzdf5H6Gk"
   },
   "source": [
    "## 6.2 图像卷积"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U_ATWo-vH6Gk"
   },
   "source": [
    "### 练习 6.2.1  \n",
    "\n",
    "构建一个具有对角线边缘的图像`X`。\n",
    "1. 如果将本节中举例的卷积核`K`应用于`X`，会发生什么情况？\n",
    "2. 如果转置`X`会发生什么？\n",
    "3. 如果转置`K`会发生什么？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tBApQWHKH6Gk"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lkEMrGV4SCW5"
   },
   "source": [
    "**第1问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kq6MSSR8H6Gk"
   },
   "source": [
    "&emsp;&emsp;在对角线处有分别为 $1$ 和 $-1$ 的数据，其他区域都为 $0$。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "hiEfX0AoH6Gl",
    "outputId": "37bc785e-2ef0-44d2-d047-0c732d1cb81e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 1.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
      "        [-1.,  1.,  0.,  0.,  0.,  0.,  0.],\n",
      "        [ 0., -1.,  1.,  0.,  0.,  0.,  0.],\n",
      "        [ 0.,  0., -1.,  1.,  0.,  0.,  0.],\n",
      "        [ 0.,  0.,  0., -1.,  1.,  0.,  0.],\n",
      "        [ 0.,  0.,  0.,  0., -1.,  1.,  0.],\n",
      "        [ 0.,  0.,  0.,  0.,  0., -1.,  1.],\n",
      "        [ 0.,  0.,  0.,  0.,  0.,  0., -1.]])\n"
     ]
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "def corr2d(X, K):  #@save\n",
    "    \"\"\"计算二维互相关运算\"\"\"\n",
    "    h, w = K.shape  # 获取卷积核的高度和宽度\n",
    "    Y = torch.zeros((X.shape[0] - h + 1, X.shape[1] - w + 1))  # 初始化输出矩阵 Y，大小为 (X行数 - K行数 + 1) x (X列数 - K列数 + 1)\n",
    "    for i in range(Y.shape[0]):  # 遍历 Y 的行数\n",
    "        for j in range(Y.shape[1]):  # 遍历 Y 的列数\n",
    "            Y[i, j] = (X[i:i + h, j:j + w] * K).sum()  # 在 Y 的每个位置，计算 X 与 K 的互相关运算结果\n",
    "    return Y\n",
    "\n",
    "# 创建一个8x8的单位矩阵 X\n",
    "X = torch.eye(8)\n",
    "# 创建一个1x2的卷积核 K，该卷积核在X上进行互相关运算\n",
    "K = torch.tensor([[1.0, -1.0]])\n",
    "# 调用corr2d函数，计算互相关运算结果Y\n",
    "Y = corr2d(X, K)\n",
    "# 打印互相关运算结果Y\n",
    "print(Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "e3Oz9yYDSdI3"
   },
   "source": [
    "**第2问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mUbUM7rZH6Gl"
   },
   "source": [
    "&emsp;&emsp;`X`转置后，结果不变。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "02INn5CxH6Gl",
    "outputId": "8841140a-0258-40f5-a0be-1f9306ef7b60"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 1.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
      "        [-1.,  1.,  0.,  0.,  0.,  0.,  0.],\n",
      "        [ 0., -1.,  1.,  0.,  0.,  0.,  0.],\n",
      "        [ 0.,  0., -1.,  1.,  0.,  0.,  0.],\n",
      "        [ 0.,  0.,  0., -1.,  1.,  0.,  0.],\n",
      "        [ 0.,  0.,  0.,  0., -1.,  1.,  0.],\n",
      "        [ 0.,  0.,  0.,  0.,  0., -1.,  1.],\n",
      "        [ 0.,  0.,  0.,  0.,  0.,  0., -1.]])\n"
     ]
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "def corr2d(X, K):  #@save\n",
    "    \"\"\"计算二维互相关运算\"\"\"\n",
    "    h, w = K.shape  # 获取卷积核的高度和宽度\n",
    "    Y = torch.zeros((X.shape[0] - h + 1, X.shape[1] - w + 1))  # 初始化输出矩阵 Y，大小为 (X行数 - K行数 + 1) x (X列数 - K列数 + 1)\n",
    "    for i in range(Y.shape[0]):  # 遍历 Y 的行数\n",
    "        for j in range(Y.shape[1]):  # 遍历 Y 的列数\n",
    "            Y[i, j] = (X[i:i + h, j:j + w] * K).sum()  # 在 Y 的每个位置，计算 X 与 K 的互相关运算结果\n",
    "    return Y\n",
    "\n",
    "# 创建一个8x8的单位矩阵 X\n",
    "X = torch.eye(8)\n",
    "# 创建一个1x2的卷积核 K，该卷积核在X的转置上进行互相关运算\n",
    "K = torch.tensor([[1.0, -1.0]])\n",
    "# 调用corr2d函数，计算互相关运算结果Y\n",
    "Y = corr2d(X.T, K) # 在 X 的转置上应用卷积核 K，得到卷积响应图像 Y\n",
    "# 打印互相关运算结果 Y\n",
    "print(Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RWwEZaw0StRK"
   },
   "source": [
    "**第3问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MTOiA5ipH6Gl"
   },
   "source": [
    "&emsp;&emsp;`K`转置后，结果也会发生转置。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "yD4k1hrmH6Gl",
    "outputId": "6502c42f-12b4-4260-fc74-f1987a55f870"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 1., -1.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
      "        [ 0.,  1., -1.,  0.,  0.,  0.,  0.,  0.],\n",
      "        [ 0.,  0.,  1., -1.,  0.,  0.,  0.,  0.],\n",
      "        [ 0.,  0.,  0.,  1., -1.,  0.,  0.,  0.],\n",
      "        [ 0.,  0.,  0.,  0.,  1., -1.,  0.,  0.],\n",
      "        [ 0.,  0.,  0.,  0.,  0.,  1., -1.,  0.],\n",
      "        [ 0.,  0.,  0.,  0.,  0.,  0.,  1., -1.]])\n"
     ]
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "def corr2d(X, K):  #@save\n",
    "    \"\"\"计算二维互相关运算\"\"\"\n",
    "    h, w = K.shape  # 获取卷积核的高度和宽度\n",
    "    Y = torch.zeros((X.shape[0] - h + 1, X.shape[1] - w + 1))  # 初始化输出矩阵 Y，大小为 (X行数 - K行数 + 1) x (X列数 - K列数 + 1)\n",
    "    for i in range(Y.shape[0]):  # 遍历 Y 的行数\n",
    "        for j in range(Y.shape[1]):  # 遍历 Y 的列数\n",
    "            Y[i, j] = (X[i:i + h, j:j + w] * K).sum()  # 在 Y 的每个位置，计算 X 与 K 的互相关运算结果\n",
    "    return Y\n",
    "\n",
    "# 创建一个8x8的单位矩阵 X\n",
    "X = torch.eye(8)\n",
    "# 创建一个1x2的卷积核 K，将其转置后与 X 进行互相关运算\n",
    "K = torch.tensor([[1.0, -1.0]])\n",
    "# 调用 corr2d 函数，计算互相关运算结果 Y\n",
    "Y = corr2d(X, K.T) # 在 X 应用卷积核 K 的转置，得到卷积响应图像 Y\n",
    "# 打印互相关运算结果 Y\n",
    "print(Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Jxhbn1TzH6Gl"
   },
   "source": [
    "### 练习 6.2.2 \n",
    "\n",
    "在我们创建的`Conv2D`自动求导时，有什么错误消息？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tZh5dWTzH6Gl"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FsZJjjLoH6Gl"
   },
   "source": [
    "&emsp;&emsp;会提示维度不对称的错误信息，因为`torch`提供的二维卷积层是`nn.Conv2d()` 采用的是四维输入和输出格式（批量大小、通道、高度、宽度），而我们自定义的仅仅是二维的。\n",
    "\n",
    "&emsp;&emsp;使用`nn.Conv2d()`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 2, loss 8.297\n",
      "epoch 4, loss 1.864\n",
      "epoch 6, loss 0.506\n",
      "epoch 8, loss 0.164\n",
      "epoch 10, loss 0.060\n"
     ]
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义二维互相关运算函数\n",
    "def corr2d(X, K):\n",
    "    \"\"\"计算二维互相关运算\"\"\"\n",
    "    h, w = K.shape  # 获取卷积核的高度和宽度\n",
    "    Y = torch.zeros((X.shape[0] - h + 1, X.shape[1] - w + 1))  # 初始化输出矩阵Y\n",
    "    for i in range(Y.shape[0]):  # 遍历输出矩阵的每一行\n",
    "        for j in range(Y.shape[1]):  # 遍历输出矩阵的每一列\n",
    "            Y[i, j] = (X[i:i + h, j:j + w] * K).sum()  # 执行互相关运算并赋值给Y的相应位置\n",
    "    return Y\n",
    "\n",
    "# 定义一个二维卷积层类\n",
    "class Conv2D(nn.Module):\n",
    "    def __init__(self, kernel_size):\n",
    "        super().__init__()  # 调用nn.Module的构造函数\n",
    "        self.weight = nn.Parameter(torch.rand(kernel_size))  # 初始化卷积核参数\n",
    "        self.bias = nn.Parameter(torch.zeros(1))  # 初始化偏置\n",
    "\n",
    "    def forward(self, x):  # 定义前向传播函数\n",
    "        return corr2d(x, self.weight) + self.bias  # 返回卷积运算结果\n",
    "\n",
    "# 创建一个6x8的矩阵，中间4列为0\n",
    "X = d2l.ones((6, 8))\n",
    "X[:, 2:6] = 0\n",
    "\n",
    "# 创建一个1x2的卷积核\n",
    "K = d2l.tensor([[1.0, -1.0]])\n",
    "\n",
    "# 进行卷积运算\n",
    "Y = corr2d(X, K)\n",
    "\n",
    "# 对X进行转置，然后再次进行卷积运算\n",
    "corr2d(d2l.transpose(X), K)\n",
    "\n",
    "# 创建一个卷积层，具有1个输出通道和（1，2）形状的卷积核\n",
    "conv2d = nn.Conv2d(1, 1, kernel_size=(1, 2), bias=False)\n",
    "\n",
    "# 调整X和Y的形状以符合PyTorch卷积层的输入输出要求\n",
    "X = X.reshape((1, 1, 6, 8))\n",
    "Y = Y.reshape((1, 1, 6, 7))\n",
    "\n",
    "# 设置学习率\n",
    "lr = 3e-2  \n",
    "\n",
    "# 尝试进行10次迭代\n",
    "try:\n",
    "    for i in range(10):\n",
    "        Y_hat = conv2d(X)  # 前向传播\n",
    "        l = (Y_hat - Y) ** 2  # 计算损失\n",
    "        conv2d.zero_grad()  # 梯度清零\n",
    "        l.sum().backward()  # 反向传播\n",
    "        conv2d.weight.data[:] -= lr * conv2d.weight.grad  # 更新权重\n",
    "        if (i + 1) % 2 == 0:  # 每两个epoch打印一次损失值\n",
    "            print(f'epoch {i+1}, loss {l.sum():.3f}')\n",
    "except Exception as e:  # 捕获并打印任何异常\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CPIxue-DxOx-"
   },
   "source": [
    "&emsp;&emsp;使用创建的`Conv2D`时可以会报如下错误。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "msTn2YsGDq87",
    "outputId": "83bafb8a-8ffe-42fb-ccb5-775efe42b095"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The size of tensor a (0) must match the size of tensor b (7) at non-singleton dimension 3\n"
     ]
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义二维互相关运算函数\n",
    "def corr2d(X, K):  #@save\n",
    "    \"\"\"计算二维互相关运算\"\"\"\n",
    "    h, w = K.shape  # 获取卷积核的高度和宽度\n",
    "    Y = torch.zeros((X.shape[0] - h + 1, X.shape[1] - w + 1))  # 初始化输出矩阵Y\n",
    "    for i in range(Y.shape[0]):  # 遍历输出矩阵的每一行\n",
    "        for j in range(Y.shape[1]):  # 遍历输出矩阵的每一列\n",
    "            Y[i, j] = (X[i:i + h, j:j + w] * K).sum()  # 执行互相关运算并赋值给Y的相应位置\n",
    "    return Y\n",
    "\n",
    "# 定义一个二维卷积层类\n",
    "class Conv2D(nn.Module):\n",
    "    def __init__(self, kernel_size):\n",
    "        super().__init__()  # 调用nn.Module的构造函数\n",
    "        self.weight = nn.Parameter(torch.rand(kernel_size))  # 初始化卷积核参数\n",
    "        self.bias = nn.Parameter(torch.zeros(1))  # 初始化偏置\n",
    "\n",
    "    def forward(self, x):  # 定义前向传播函数\n",
    "        return corr2d(x, self.weight) + self.bias  # 返回卷积运算结果\n",
    "\n",
    "# 创建一个6x8的矩阵，中间4列为0\n",
    "X = d2l.ones((6, 8))\n",
    "X[:, 2:6] = 0\n",
    "\n",
    "# 创建一个1x2的卷积核\n",
    "K = d2l.tensor([[1.0, -1.0]])\n",
    "\n",
    "# 使用corr2d函数进行卷积运算\n",
    "Y = corr2d(X, K)\n",
    "\n",
    "# 对X进行转置，然后再次使用corr2d函数进行卷积运算\n",
    "corr2d(d2l.transpose(X), K)\n",
    "\n",
    "# 创建一个卷积层，具有1个输出通道和（1，2）形状的卷积核\n",
    "conv2d = Conv2D(kernel_size=(1, 2))\n",
    "\n",
    "# 调整X和Y的形状以符合PyTorch卷积层的输入输出要求\n",
    "X = X.reshape((1, 1, 6, 8))\n",
    "Y = Y.reshape((1, 1, 6, 7))\n",
    "\n",
    "# 设置学习率\n",
    "lr = 3e-2  \n",
    "\n",
    "# 尝试进行10次迭代\n",
    "try:\n",
    "    for i in range(10):\n",
    "        Y_hat = conv2d(X)  # 前向传播\n",
    "        l = (Y_hat - Y) ** 2  # 计算损失\n",
    "        conv2d.zero_grad()  # 梯度清零\n",
    "        l.sum().backward()  # 反向传播\n",
    "        conv2d.weight.data[:] -= lr * conv2d.weight.grad  # 更新权重\n",
    "        if (i + 1) % 2 == 0:  # 每两个epoch打印一次损失值\n",
    "            print(f'epoch {i+1}, loss {l.sum():.3f}')\n",
    "except Exception as e:  # 捕获并打印任何异常\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Gax6VocXxTnp"
   },
   "source": [
    "&emsp;&emsp;可以通过下述方式调整。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "SMFz77vLEcrd",
    "outputId": "650e4729-b223-4fb7-9bd9-429b4dcb77eb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 2, loss 20.407\n",
      "epoch 4, loss 7.008\n",
      "epoch 6, loss 2.644\n",
      "epoch 8, loss 1.045\n",
      "epoch 10, loss 0.422\n"
     ]
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义一个函数用于计算二维互相关运算\n",
    "def corr2d(X, K):  #@save\n",
    "    \"\"\"计算二维互相关运算\"\"\"\n",
    "    h, w = K.shape  # 获取卷积核K的高度和宽度\n",
    "    Y = torch.zeros((X.shape[0] - h + 1, X.shape[1] - w + 1))  # 初始化输出矩阵Y\n",
    "    for i in range(Y.shape[0]):  # 对输出矩阵的行进行遍历\n",
    "        for j in range(Y.shape[1]):  # 对输出矩阵的列进行遍历\n",
    "            Y[i, j] = (X[i:i + h, j:j + w] * K).sum()  # 进行元素乘法并求和，计算互相关\n",
    "    return Y\n",
    "\n",
    "# 定义一个二维卷积层类\n",
    "class Conv2D(nn.Module):\n",
    "    def __init__(self, kernel_size):\n",
    "        super().__init__()  # 调用超类的构造函数\n",
    "        self.weight = nn.Parameter(torch.rand(kernel_size))  # 随机初始化卷积核的权重\n",
    "        self.bias = nn.Parameter(torch.zeros(1))  # 初始化偏置为0\n",
    "\n",
    "    def forward(self, x):  # 定义前向传播的计算\n",
    "        return corr2d(x, self.weight) + self.bias  # 进行卷积运算并加上偏置\n",
    "\n",
    "# 创建一个6x8的矩阵，中间4列设置为0\n",
    "X = d2l.ones((6, 8))\n",
    "X[:, 2:6] = 0\n",
    "\n",
    "# 创建一个1x2的卷积核\n",
    "K = d2l.tensor([[1.0, -1.0]])\n",
    "\n",
    "# 使用corr2d函数进行卷积运算\n",
    "Y = corr2d(X, K)\n",
    "\n",
    "# 对X进行转置，然后再次使用corr2d函数进行卷积运算\n",
    "corr2d(d2l.transpose(X), K)\n",
    "\n",
    "# 创建一个卷积层，具有1个输出通道和（1，2）形状的卷积核\n",
    "conv2d = Conv2D(kernel_size=(1, 2))\n",
    "\n",
    "# 重新调整X和Y的形状，以符合卷积层的输入要求\n",
    "X = X.reshape((6, 8))\n",
    "Y = Y.reshape((6, 7))\n",
    "\n",
    "# 设置学习率\n",
    "lr = 3e-2\n",
    "\n",
    "# 尝试进行10次迭代\n",
    "try:\n",
    "    for i in range(10):\n",
    "        Y_hat = conv2d(X)  # 前向传播\n",
    "        l = (Y_hat - Y) ** 2  # 计算损失\n",
    "        conv2d.zero_grad()  # 梯度清零\n",
    "        l.sum().backward()  # 反向传播\n",
    "        conv2d.weight.data[:] -= lr * conv2d.weight.grad  # 更新权重\n",
    "        if (i + 1) % 2 == 0:  # 每两个epoch打印一次损失值\n",
    "            print(f'epoch {i+1}, loss {l.sum():.3f}')\n",
    "except Exception as e:  # 捕获并打印任何异常\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_rl_95PQH6Gm"
   },
   "source": [
    "### 练习 6.2.3\n",
    "\n",
    "如何通过改变输入张量和卷积核张量，将互相关运算表示为矩阵乘法？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iqzn_dbiH6Gm"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KzbnEauZH6Gm"
   },
   "source": [
    "&emsp;&emsp;题目的意思应该是如何通过矩阵乘法得到互相关（卷积）运算。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "vIojUxrQH6Gm"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 0., 1.],\n",
      "        [1., 0., 1.]])\n",
      "tensor([[4.]])\n"
     ]
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义一个函数来通过矩阵乘法实现二维卷积操作\n",
    "def conv2d_by_mul(X, K):\n",
    "    # 获取卷积核大小\n",
    "    h, w = K.shape\n",
    "    # 计算输出图像大小\n",
    "    outh = X.shape[0] - h + 1\n",
    "    outw = X.shape[1] - w + 1\n",
    "    # 调整卷积核形状以便做乘法\n",
    "    K = K.reshape(-1, 1)\n",
    "    # 将输入图像切成卷积核大小的块，打平成一维，存放在列表 Y 中\n",
    "    Y = []\n",
    "    for i in range(outh):\n",
    "        for j in range(outw):\n",
    "            Y.append(X[i:i + h, j:j + w].reshape(-1))\n",
    "    # 将列表 Y 转为张量，每行代表一块的打平结果\n",
    "    Y = torch.stack(Y, 0)\n",
    "    # 用矩阵乘法表示互相关运算\n",
    "    res = (torch.matmul(Y, K)).reshape(outh, outw)\n",
    "    # 返回输出结果\n",
    "    return res\n",
    "\n",
    "# 示例\n",
    "# 创建一个2x3的输入图像 X，其中第二列的元素被设置为0\n",
    "X = torch.ones((2, 3))\n",
    "X[:, 1] = 0  # 将第二列的所有元素设置为0\n",
    "# 打印输入图像 X\n",
    "print(X)\n",
    "# 创建一个2x3的卷积核 K，所有元素均为1\n",
    "K = torch.ones((2, 3))\n",
    "# 调用自定义的 conv2d_by_mul 函数来执行二维互相关运算\n",
    "output = conv2d_by_mul(X, K)\n",
    "# 打印互相关运算的结果\n",
    "print(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0Ta-sCxFH6Gm"
   },
   "source": [
    "### 练习 6.2.4\n",
    "\n",
    "手工设计一些卷积核。\n",
    "1. 二阶导数的核的形式是什么？\n",
    "1. 积分的核的形式是什么？\n",
    "1. 得到$d$次导数的最小核的大小是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2TpbNlmoH6Gm"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Sj92on4zTnmx"
   },
   "source": [
    "**第1问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZFTY9ib1Tr6j"
   },
   "source": [
    "&emsp;&emsp;一维的二阶导数的核的形式是：\n",
    "\n",
    "$$\\begin{bmatrix}-1, & 2, & -1,\\end{bmatrix}$$\n",
    "\n",
    "&emsp;&emsp;这个一维卷积核可以用于计算信号或函数在每个点的二阶导数。它通过对信号进行两次微分来强调信号的曲率和变化率，通常用于边缘检测和特征检测等任务。\n",
    "\n",
    "&emsp;&emsp;在二维情况下，可以使用以下卷积核来计算图像的二阶导数：\n",
    "\n",
    "\\begin{bmatrix}\n",
    "    0, &  1, & 0 \\\\\n",
    "    1, & -4, & 1 \\\\\n",
    "    0, &  1, & 0 \\\\\n",
    "\\end{bmatrix}\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZjHoDCPDTwf8"
   },
   "source": [
    "**第2问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8-kf1IxsH6Gm"
   },
   "source": [
    "&emsp;&emsp;积分的核的形式是：\n",
    "\n",
    "$$\\begin{bmatrix}1 & 1 & 1 & \\cdots & 1\\end{bmatrix}$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "20GNRTaGT3vl"
   },
   "source": [
    "**第3问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mLPsGuj_T560"
   },
   "source": [
    "&emsp;&emsp;得到 `𝑑` 次导数的最小核的大小是 $d+1$。例如，一阶导数的最小核大小为 $2$，二阶导数的最小核大小为 $3$，三阶导数的最小核大小为 $4$，以此类推。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Xxl6bZHSH6Gm"
   },
   "source": [
    "## 6.3 填充和步幅 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "F4I1FfTtH6Gm"
   },
   "source": [
    "### 练习 6.3.1  \n",
    "\n",
    "对于本节中的最后一个示例，计算其输出形状，以查看它是否与实验结果一致。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "db2l2ZVaH6Gm"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aUp1JW-cH6Gm"
   },
   "source": [
    "$$out_{shape}=\\lfloor(n_h-k_h+p_h*2+s_h)/s_h\\rfloor \\times \\lfloor(n_w-k_w+p_w*2+s_w)/s_w\\rfloor.$$\n",
    "\n",
    "&emsp;&emsp;示例中 `X.shape = [8, 8]` ，计算得出 $out_shape = [(8-3+0+3)/3, (8-5+2+4)/4] = [2.67, 2.25]$ ，向下取整，所以为 $[2, 2]$。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Un2WNAxXzVvO",
    "outputId": "caf4dd00-192e-4cbb-def6-8dc86a3a632c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([2, 2])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch库\n",
    "import torch\n",
    "from torch import nn\n",
    "\n",
    "# 定义一个函数来计算卷积层的输出\n",
    "def comp_conv2d(conv2d, X):\n",
    "    # 这里的（1，1）表示批量大小和通道数都是1\n",
    "    X = X.reshape((1, 1) + X.shape)  # 重塑输入数据X的形状，以符合卷积层的输入要求（批量大小，通道数，高度，宽度）\n",
    "    Y = conv2d(X)  # 使用卷积层对输入数据X进行卷积操作\n",
    "    # 省略前两个维度：批量大小和通道\n",
    "    return Y.reshape(Y.shape[2:])  # 返回卷积操作的结果，省略批量大小和通道数维度\n",
    "\n",
    "# 请注意，这里每边都填充了1行或1列，因此总共添加了2行或2列\n",
    "X = torch.rand(size=(8, 8))  # 创建一个随机的8x8矩阵作为输入数据\n",
    "\n",
    "# 创建一个卷积层，参数包括：输入通道数1，输出通道数1，卷积核大小3x5，填充大小0x1，步幅3x4\n",
    "conv2d = nn.Conv2d(1, 1, kernel_size=(3, 5), padding=(0, 1), stride=(3, 4))\n",
    "\n",
    "# 使用comp_conv2d函数计算卷积层的输出，并打印输出的形状\n",
    "comp_conv2d(conv2d, X).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "brG_eGKcH6Gn"
   },
   "source": [
    "### 练习 6.3.2\n",
    "\n",
    "在本节中的实验中，试一试其他填充和步幅组合。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lFoUTJ82H6Gn"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n8bXitvc1p0Q"
   },
   "source": [
    "&emsp;&emsp; 下面将举两个其他的填充和步幅组合。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pJ3E2kSW1xuG"
   },
   "source": [
    "&emsp;&emsp;在`padding`大小为 $[1,2]$ ，`stride`大小为 $[2,3]$ 时，输出形状为 $[4,3]$。\n",
    "\n",
    "&emsp;&emsp;示例中 `X.shape = [8, 8]`，计算得出 $out_{shape} = [(8-3+2+2)/2, (8-5+4+3)/3] = [4.5, 3,33]$ ，向下取整，所以为 $[4, 3]$。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "0RMAerR41NZk",
    "outputId": "e84b746f-5981-4e7d-a571-2cfb8dd7beda"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([4, 3])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch库\n",
    "import torch\n",
    "from torch import nn\n",
    "\n",
    "# 定义一个函数来计算卷积层的输出\n",
    "def comp_conv2d(conv2d, X):\n",
    "    # 这里的（1，1）表示批量大小和通道数都是1\n",
    "    X = X.reshape((1, 1) + X.shape)  # 重塑输入数据X的形状，以符合卷积层的输入要求（批量大小，通道数，高度，宽度）\n",
    "    Y = conv2d(X)  # 使用卷积层对输入数据X进行卷积操作\n",
    "    # 省略前两个维度：批量大小和通道\n",
    "    return Y.reshape(Y.shape[2:])  # 返回卷积操作的结果，省略批量大小和通道数维度\n",
    "\n",
    "# 创建一个随机的8x8矩阵作为输入数据\n",
    "X = torch.rand(size=(8, 8))\n",
    "\n",
    "# 创建一个卷积层，参数包括：输入通道数1，输出通道数1，卷积核大小3x5，填充大小1x2，步幅2x3\n",
    "conv2d = nn.Conv2d(1, 1, kernel_size=(3, 5), padding=(1, 2), stride=(2, 3))\n",
    "\n",
    "# 使用comp_conv2d函数计算卷积层的输出，并打印输出的形状\n",
    "comp_conv2d(conv2d, X).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3KVUq8Zu4Okx"
   },
   "source": [
    "&emsp;&emsp;在`padding`大小为 $[2,3]$ ，`stride`大小为 $[1,2]$ 时，输出形状为 $[10,5]$ 。\n",
    "\n",
    "&emsp;&emsp;示例中 `X.shape = [8, 8]` ，计算得出 $out_{shape} = [(8-3+4+1)/1, (8-5+6+2)/2] = [10, 5.5]$ ，向下取整，所以为 $[10, 5]$ 。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "dbrqAE0G1bok",
    "outputId": "217140fc-29d4-45a5-eb18-c69ead1c1ef6"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([10, 5])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch库\n",
    "import torch\n",
    "from torch import nn\n",
    "\n",
    "# 定义一个函数来计算卷积层的输出\n",
    "def comp_conv2d(conv2d, X):\n",
    "    # 这里的（1，1）表示批量大小和通道数都是1\n",
    "    X = X.reshape((1, 1) + X.shape)  # 重塑输入X的形状以符合卷积层的输入要求（批量大小，通道数，高度，宽度）\n",
    "    Y = conv2d(X)  # 对输入X应用卷积层\n",
    "    # 省略前两个维度：批量大小和通道\n",
    "    return Y.reshape(Y.shape[2:])  # 去除批量大小和通道数维度，返回卷积结果\n",
    "\n",
    "# 创建一个随机的8x8矩阵作为输入数据\n",
    "X = torch.rand(size=(8, 8))\n",
    "\n",
    "# 创建一个卷积层，参数包括：输入通道数1，输出通道数1，卷积核大小3x5，填充大小2x3，步幅1x2\n",
    "conv2d = nn.Conv2d(1, 1, kernel_size=(3, 5), padding=(2, 3), stride=(1, 2))\n",
    "\n",
    "# 使用comp_conv2d函数计算卷积层的输出，并打印输出的形状\n",
    "comp_conv2d(conv2d, X).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uTl-5K1SH6Gn"
   },
   "source": [
    "### 练习 6.3.3\n",
    "\n",
    "对于音频信号，步幅$2$说明什么？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YNXFYzlhH6Gn"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nuueFr5_H6Gn"
   },
   "source": [
    "&emsp;&emsp;对于音频信号而言，步幅为 $2$ 就是以 $2$ 为周期对信号进行采样计算。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_3anPQxXH6Gn"
   },
   "source": [
    "### 练习 6.3.4\n",
    "\n",
    "步幅大于$1$的计算优势是什么？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "q02rwtZnH6Gn"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7_po2mkiH6Gn"
   },
   "source": [
    "&emsp;&emsp;1.**减少计算量**：步幅大于1意味着卷积核在输入特征图（`feature map`）上的滑动距离更大，因此每次卷积操作涵盖的输入数据更少。这减少了进行卷积操作所需的计算量。\\\n",
    "&emsp;&emsp;2.**降低空间维度**：使用大步幅会减小输出特征图的空间尺寸。例如，在进行下采样（`subsampling`）或降维时，较大的步幅可以有效减小输出数据的高度和宽度，这有助于减少后续层的计算负担。\\\n",
    "&emsp;&emsp;3.**控制过拟合**：通过减少网络中的参数数量和计算量，大步幅有助于控制过拟合。这是因为减少了学习参数，从而降低了模型复杂度。\\\n",
    "&emsp;&emsp;4.**增加感受野**：较大的步幅可以增加网络层的感受野（即网络层可以观察到的输入数据的区域大小）。这意味着网络可以在较低的空间分辨率下捕捉到更广泛的上下文信息。\\\n",
    "&emsp;&emsp;5.**加快网络训练速度**：由于计算量的减少和参数数量的降低，使用较大步幅可以加快网络的训练速度，使得模型更快地收敛。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rLzkTd9KH6Gn"
   },
   "source": [
    "## 6.4 多输入多输出通道 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "H0Acj8q_H6Gn"
   },
   "source": [
    "### 练习 6.4.1\n",
    "\n",
    "假设我们有两个卷积核，大小分别为 $k_1$ 和 $k_2$（中间没有非线性激活函数）。\n",
    "1. 证明运算可以用单次卷积来表示。\n",
    "2. 这个等效的单个卷积核的维数是多少呢？\n",
    "3. 反之亦然吗？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-WvYPWNzH6Go"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hCHo1nmiW269"
   },
   "source": [
    "**第1问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J0WKW_2rH6Go"
   },
   "source": [
    "&emsp;&emsp;假设输入的图像大小为 $W×H$ ，设卷积核 $1$ 的大小为 $k_1$ ，卷积核 $2$ 的大小为 $k2$ ，它们分别作用于输入矩阵 $x$ ，得到的输出矩阵分别为 $y_1$ 和 $y_2$ 。则可以将 $y1$ 与 $y2$ 的每一个元素相加，得到最终输出矩阵 $y$。\n",
    "\n",
    "&emsp;&emsp;即：$$y[i][j] = y_1[i][j] + y_2[i][j]$$\n",
    "\n",
    "&emsp;&emsp;可以将两个卷积核的大小相加，得到一个新的卷积核大小为 $(k_1+k_2-1)×(k_1+k_2-1)$ 。然后可以将这个新的卷积核应用于输入图像，得到一个输出图像。这个输出图像的大小为 $(W-k_1-k_2+2)×(H-k_1-k_2+2)$ 。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "a_oG7ejjZIts"
   },
   "source": [
    "**第2问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Iu5hDtACZI_Q"
   },
   "source": [
    "&emsp;&emsp;可以使用一个大小为 $(k_1+k_2-1)×(k_1+k_2-1)$ 的卷积核来表示这两个卷积核的运算。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ucb3Vc2pZJUL"
   },
   "source": [
    "**第3问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sKoiH9o-ZJbF"
   },
   "source": [
    "&emsp;&emsp;反之亦然。如果有一个大小为 $k_1+k_2-1$ 的卷积核，可以将其分解为两个大小分别为 $k1$ 和 $k2$ 的卷积核。这两个卷积核之间没有非线性激活函数，所以它们的运算可以被视为一个单独的卷积核。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "I5H7cCLpH6Go"
   },
   "source": [
    "### 练习 6.4.2\n",
    "\n",
    "假设输入为 $c_i\\times h\\times w$ ，卷积核大小为 $c_o\\times c_i\\times k_h\\times k_w$ ，填充为 $(p_h, p_w)$，步幅为$(s_h, s_w)$。\n",
    "1. 前向传播的计算成本（乘法和加法）是多少？\n",
    "1. 内存占用是多少？\n",
    "1. 反向传播的内存占用是多少？\n",
    "1. 反向传播的计算成本是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3a-nXFFtH6Go"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WPC8XK92bNMD"
   },
   "source": [
    "**第1问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GlNurvDQbNQ0"
   },
   "source": [
    "&emsp;&emsp;前向计算成本为\n",
    "\n",
    "$$flops_{forward} = c_i \\times c_o \\times k_h \\times k_w \\times m_h \\times m_w$$\n",
    "\n",
    "&emsp;&emsp;其中 $m_h=\\lfloor \\frac{h+2p_h-k_h}{s_h}+1 \\rfloor$ , $m_w=\\lfloor \\frac{w+2p_w-k_w}{s_w}+1 \\rfloor$ 。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J_oix-mgbNZ3"
   },
   "source": [
    "**第2问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zh1SQkZbbNhz"
   },
   "source": [
    "&emsp;&emsp;输入需要 $c_i*h*w$ 个浮点数，卷积核需要 $c_o*c_i*k_h*k_w$ 个浮点数，输出需要 $c_o*m_h*m_w$ 个浮点数。此外，还需要存储中间结果，即填充后的输入和反向传播时的梯度信息。因此，总内存占用为 \n",
    "$$memory_{forward}=(c_i+k_h-1)*(h+k_w-1)*c_0+2*c_i*h*w$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JMGABMnGbNrB"
   },
   "source": [
    "**第3问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9Ef0fAhubNy-"
   },
   "source": [
    "&emsp;&emsp;反向传播的内存作用与前向传播相同，总内存占用为\n",
    "$$memory_{backward}=(c_i+k_h-1)*(h+k_w-1)*c_0+2*c_i*h*w$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XqGjYQ5LbN7L"
   },
   "source": [
    "**第4问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9JrDsP6pbOCT"
   },
   "source": [
    "&emsp;&emsp;反向计算成本为\n",
    "\n",
    "$$flops_{backward} = c_i \\times c_o \\times k_h \\times k_w \\times m_h \\times m_w$$\n",
    "\n",
    "&emsp;&emsp;其中 $m_h$ 和 $m_w$ 的定义同上。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qKKQr0MPH6Go"
   },
   "source": [
    "### 练习 6.4.3\n",
    "\n",
    "如果我们将输入通道 $c_i$ 和输出通道 $c_o$ 的数量加倍，计算数量会增加多少？如果我们把填充数量翻一番会怎么样？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "N825fzRxH6Go"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CifxEEGjH6Go"
   },
   "source": [
    "&emsp;&emsp;如果我们将输入通道 $c_i$ 和输出通道 $c_o$ 的数量加倍，计算数量会增加 $4$ 倍。如果我们把填充数量翻一番，计算数量会增加 $2$ 倍。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PLWGRhpQH6Go"
   },
   "source": [
    "### 练习 6.4.4\n",
    "\n",
    "如果卷积核的高度和宽度是 $k_h=k_w=1$ ，前向传播的计算复杂度是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4MBYJCXuH6Go"
   },
   "source": [
    "**解答：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j4Wcrl3XH6Gp"
   },
   "source": [
    "$$flops = c_i \\times c_o \\times \\frac{h-p_h}{s_h} \\times \\frac{w-p_w}{s_w}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "khZEzrrsH6Gp"
   },
   "source": [
    "### 练习 6.4.5\n",
    "\n",
    "本节最后一个示例中的变量 $Y1$ 和 $Y2$ 是否完全相同？为什么？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "D4amqFuFH6Gp"
   },
   "source": [
    "**解答：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "K9YjhzTfH6Gp"
   },
   "source": [
    "&emsp;&emsp;理论上相同，但由于浮点数计算有误差，因而两者不完全相同（有微小差异）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HGIsrbxRH6Gp"
   },
   "source": [
    "### 练习 6.4.6\n",
    "\n",
    "当卷积窗口不是 $1\\times 1$ 时，如何使用矩阵乘法实现卷积？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LHRBB75GH6Gp"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-JowXZuVH6Gp"
   },
   "source": [
    "&emsp;&emsp;可以将输入张量和卷积核张量分别展开为二维矩阵，然后对这两个矩阵进行乘法运算，得到的结果再变换为输出张量。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LQluJheeH6Gp"
   },
   "source": [
    "## 6.5 汇聚层 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WXmo99OwH6Gp"
   },
   "source": [
    "### 练习 6.5.1\n",
    "\n",
    "尝试将平均汇聚层作为卷积层的特殊情况实现。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VQDuZsKZH6Gq"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "vpS7o9ZjH6Gq"
   },
   "outputs": [],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch库中的必要组件\n",
    "import torch.nn as nn \n",
    "import torch.nn.functional as F\n",
    "\n",
    "# 定义一个继承自nn.Module的神经网络类\n",
    "class Net(nn.Module): \n",
    "    def __init__(self):\n",
    "        # 调用父类的初始化方法\n",
    "        super(Net, self).__init__()\n",
    "        # 定义卷积层1，输入通道数为1，输出通道数为6，卷积核大小为5x5\n",
    "        self.conv1 = nn.Conv2d(1, 6, 5)\n",
    "        # 定义池化层，将6个通道的特征图进行平均池化\n",
    "        self.pool = nn.Conv2d(6, 6, 5)\n",
    "        # 定义卷积层2，输入通道数为6，输出通道数为16，卷积核大小为5x5\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        # 定义全连接层1，输入大小为16*5*5，输出大小为120\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
    "        # 定义全连接层2，输入大小为120，输出大小为84\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        # 定义全连接层3，输入大小为84，输出大小为10（对应10个类别的分类）\n",
    "        self.fc3 = nn.Linear(84, 10)\n",
    "    \n",
    "    # 定义网络的前向传播路径\n",
    "    def forward(self, x):\n",
    "        # 使用ReLU激活函数进行卷积层1的前向传播\n",
    "        x = F.relu(self.conv1(x))\n",
    "        # 使用平均池化进行池化层的前向传播\n",
    "        x = F.avg_pool2d(x, (2, 2))\n",
    "        # 使用ReLU激活函数进行卷积层2的前向传播\n",
    "        x = F.relu(self.conv2(x))\n",
    "        # 使用平均池化进行池化层的前向传播\n",
    "        x = F.avg_pool2d(x, (2, 2))\n",
    "        # 将特征图展平为一维向量\n",
    "        x = x.view(-1, self.num_flat_features(x))\n",
    "        # 使用ReLU激活函数进行全连接层1的前向传播\n",
    "        x = F.relu(self.fc1(x))\n",
    "        # 使用ReLU激活函数进行全连接层2的前向传播\n",
    "        x = F.relu(self.fc2(x))\n",
    "        # 全连接层3的前向传播，没有使用激活函数\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "    \n",
    "    # 定义一个辅助函数来计算特征图展平后的特征数量\n",
    "    def num_flat_features(self, x):\n",
    "        size = x.size()[1:] # 获取除了批量维度之外的其他维度的大小\n",
    "        num_features = 1\n",
    "        for s in size:\n",
    "            num_features *= s  # 计算总的特征数\n",
    "        return num_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ryQl9D8qH6Gr"
   },
   "source": [
    "### 练习 6.5.2\n",
    "\n",
    "尝试将最大汇聚层作为卷积层的特殊情况实现。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-09jj2UrH6Gr"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "SQd7cXluH6Gr"
   },
   "outputs": [],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch库\n",
    "import torch.nn as nn \n",
    "import torch.nn.functional as F\n",
    "\n",
    "class Net(nn.Module): \n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        # 定义卷积层1，输入通道数为1，输出通道数为6，卷积核大小为5x5\n",
    "        self.conv1 = nn.Conv2d(1, 6, 5)\n",
    "        # 定义池化层，将6个通道的特征图进行平均池化\n",
    "        self.pool = nn.Conv2d(6, 6, 5)\n",
    "        # 定义卷积层2，输入通道数为6，输出通道数为16，卷积核大小为5x5\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        # 定义全连接层1，输入大小为16x5x5，输出大小为120\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
    "        # 定义全连接层2，输入大小为120，输出大小为84\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        # 定义全连接层3，输入大小为84，输出大小为10（对应10个类别的分类）\n",
    "        self.fc3 = nn.Linear(84, 10)\n",
    "    \n",
    "    def forward(self, x):\n",
    "        # 使用ReLU激活函数进行卷积层1的前向传播\n",
    "        x = F.relu(self.conv1(x))\n",
    "        # 使用最大池化进行池化层的前向传播\n",
    "        x = F.max_pool2d(x, (2, 2))\n",
    "        # 使用ReLU激活函数进行卷积层2的前向传播\n",
    "        x = F.relu(self.conv2(x))\n",
    "        # 使用最大池化进行池化层的前向传播\n",
    "        x = F.max_pool2d(x, (2, 2))\n",
    "        # 将特征图展平为一维向量\n",
    "        x = x.view(-1, self.num_flat_features(x))\n",
    "        # 使用ReLU激活函数进行全连接层1的前向传播\n",
    "        x = F.relu(self.fc1(x))\n",
    "        # 使用ReLU激活函数进行全连接层2的前向传播\n",
    "        x = F.relu(self.fc2(x))\n",
    "        # 全连接层3的前向传播，没有使用激活函数\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "    \n",
    "    # 定义一个辅助函数来计算特征图展平后的特征数量\n",
    "    def num_flat_features(self, x):\n",
    "        size = x.size()[1:] # 获取除了批量维度之外的其他维度的大小\n",
    "        num_features = 1\n",
    "        for s in size:\n",
    "            num_features *= s # 计算总的特征数\n",
    "        return num_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J5YqgAbbH6Gr"
   },
   "source": [
    "### 练习 6.5.3\n",
    "\n",
    "假设汇聚层的输入大小为 $c\\times h\\times w$ ，则汇聚窗口的形状为 $p_h\\times p_w$ ，填充为 $(p_h, p_w)$ ，步幅为 $(s_h, s_w)$。这个汇聚层的计算成本是多少？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oMjxs6WbH6Gr"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ykP2lf_CH6Gr"
   },
   "source": [
    "$$flops=\\frac{c \\times h \\times w \\times p_h \\times p_w}{s_h \\times s_w}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-yiOraa6H6Gs"
   },
   "source": [
    "### 练习 6.5.4\n",
    "\n",
    "为什么最大汇聚层和平均汇聚层的工作方式不同？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6onxEYAyH6Gs"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CkRiZ35eH6Gs"
   },
   "source": [
    "&emsp;&emsp;最大池化层和平均池化层的工作方式不同，因为它们使用不同的池化方法。最大池化层将输入张量分成不重叠的区域，并在每个区域中选择最大值。平均池化层将输入张量分成不重叠的区域，并计算每个区域的平均值。这些方法的主要区别在于它们如何处理输入张量中的信息。最大池化层通常用于提取输入张量中的显著特征，而平均池化层通常用于减少输入张量的大小并提高模型的计算效率。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dE1ihMORH6Gs"
   },
   "source": [
    "### 练习 6.5.5\n",
    "\n",
    "我们是否需要最小汇聚层？可以用已知函数替换它吗？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "80MiSdFlH6Gs"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&emsp;&emsp;在神经网络中，汇聚层（`Pooling Layer`）通常用于减少特征图的空间维度，从而减少参数数量并且使网络对于平移变换更加鲁棒。常见的汇聚操作包括最大汇聚和平均汇聚。最小汇聚层并不常见，因为它没有最大汇聚和平均汇聚那样的优点，也不常用于实际的神经网络架构中。因此我们可以不需要最小汇聚层，可以对负值做最大池化操作来进行代替。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "6nbP1cklH6Gs"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[[ 0.,  1.,  2.,  3.],\n",
      "          [ 4.,  5.,  6.,  7.],\n",
      "          [ 8.,  9., 10., 11.],\n",
      "          [12., 13., 14., 15.]]]])\n",
      "tensor([[[[ 0.,  2.],\n",
      "          [ 8., 10.]]]])\n"
     ]
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch库\n",
    "import torch.nn.functional as F  # 导入PyTorch的函数模块\n",
    "\n",
    "# 定义最小池化函数\n",
    "def min_pool2d(x, kernel_size, stride=None, padding=0, dilation=1, ceil_mode=False):\n",
    "    # 将输入张量x取负数\n",
    "    neg_x = -x\n",
    "    # 使用F.max_pool2d对取负后的张量进行最大池化操作\n",
    "    neg_min_pool = F.max_pool2d(neg_x, kernel_size, stride=stride, padding=padding, dilation=dilation, ceil_mode=ceil_mode)\n",
    "    # 将经过最大池化的结果再次取负，得到最小池化的结果\n",
    "    min_pool = -neg_min_pool\n",
    "    return min_pool\n",
    "# 示例\n",
    "# 创建一个4x4的张量，值从0到15\n",
    "X = torch.arange(16, dtype=torch.float32).reshape((1, 1 ,4, 4))\n",
    "print(X)  # 打印原始张量\n",
    "print(min_pool2d(X,2))  # 应用最小池化并打印结果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1PNLI8uzH6Gt"
   },
   "source": [
    "### 练习 6.5.6\n",
    "\n",
    "除了平均汇聚层和最大汇聚层，是否有其它函数可以考虑（提示：回想一下`softmax`）？为什么它不流行？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VCnILmizH6Gt"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "O5eyNAZyxOWX"
   },
   "source": [
    "&emsp;&emsp;除了平均汇聚层和最大汇聚层，还有一些其他的池化函数，例如`Lp`池化和随机池化。`Softmax`函数通常用于多分类问题，它将每个输出分类的结果赋予一个概率值，表示属于每个类别的可能性。但是，`Softmax`函数不适用于池化层，因为它会将所有输入数据转换为概率分布，这会导致信息丢失。因此，`Softmax`函数不流行用于池化层。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J1AdlfjBH6Gt"
   },
   "source": [
    "## 6.6 卷积神经网络（LeNet） "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BmcAHKVpH6Gt"
   },
   "source": [
    "### 练习 6.6.1\n",
    "\n",
    "将平均汇聚层替换为最大汇聚层，会发生什么？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "p3D3TAOpH6Gt"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c84yG4GSH6Gt"
   },
   "source": [
    "&emsp;&emsp;输出更大，梯度更大，训练更容易（`AlexNet`改进的方式之一）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zVg6Ri3Mxxv2"
   },
   "source": [
    "&emsp;&emsp; `LeNet`原始代码如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "98dd44b7",
    "origin_pos": 2,
    "pycharm": {
     "name": "#%%\n"
    },
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.498, train acc 0.812, test acc 0.802\n",
      "65986.7 examples/sec on cuda:0\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-03-01T02:07:33.596432</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 238.965625 183.35625 \n",
       "L 238.965625 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "L 30.103125 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 51.803125 145.8 \n",
       "L 51.803125 7.2 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"mc38b9e0f66\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#mc38b9e0f66\" x=\"51.803125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(48.621875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 95.203125 145.8 \n",
       "L 95.203125 7.2 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mc38b9e0f66\" x=\"95.203125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 4 -->\n",
       "      <g transform=\"translate(92.021875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 138.603125 145.8 \n",
       "L 138.603125 7.2 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mc38b9e0f66\" x=\"138.603125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 6 -->\n",
       "      <g transform=\"translate(135.421875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
       "Q 1688 2584 1439 2293 \n",
       "Q 1191 2003 1191 1497 \n",
       "Q 1191 994 1439 701 \n",
       "Q 1688 409 2113 409 \n",
       "Q 2538 409 2786 701 \n",
       "Q 3034 994 3034 1497 \n",
       "Q 3034 2003 2786 2293 \n",
       "Q 2538 2584 2113 2584 \n",
       "z\n",
       "M 3366 4563 \n",
       "L 3366 3988 \n",
       "Q 3128 4100 2886 4159 \n",
       "Q 2644 4219 2406 4219 \n",
       "Q 1781 4219 1451 3797 \n",
       "Q 1122 3375 1075 2522 \n",
       "Q 1259 2794 1537 2939 \n",
       "Q 1816 3084 2150 3084 \n",
       "Q 2853 3084 3261 2657 \n",
       "Q 3669 2231 3669 1497 \n",
       "Q 3669 778 3244 343 \n",
       "Q 2819 -91 2113 -91 \n",
       "Q 1303 -91 875 529 \n",
       "Q 447 1150 447 2328 \n",
       "Q 447 3434 972 4092 \n",
       "Q 1497 4750 2381 4750 \n",
       "Q 2619 4750 2861 4703 \n",
       "Q 3103 4656 3366 4563 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 182.003125 145.8 \n",
       "L 182.003125 7.2 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mc38b9e0f66\" x=\"182.003125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 8 -->\n",
       "      <g transform=\"translate(178.821875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
       "Q 1584 2216 1326 1975 \n",
       "Q 1069 1734 1069 1313 \n",
       "Q 1069 891 1326 650 \n",
       "Q 1584 409 2034 409 \n",
       "Q 2484 409 2743 651 \n",
       "Q 3003 894 3003 1313 \n",
       "Q 3003 1734 2745 1975 \n",
       "Q 2488 2216 2034 2216 \n",
       "z\n",
       "M 1403 2484 \n",
       "Q 997 2584 770 2862 \n",
       "Q 544 3141 544 3541 \n",
       "Q 544 4100 942 4425 \n",
       "Q 1341 4750 2034 4750 \n",
       "Q 2731 4750 3128 4425 \n",
       "Q 3525 4100 3525 3541 \n",
       "Q 3525 3141 3298 2862 \n",
       "Q 3072 2584 2669 2484 \n",
       "Q 3125 2378 3379 2068 \n",
       "Q 3634 1759 3634 1313 \n",
       "Q 3634 634 3220 271 \n",
       "Q 2806 -91 2034 -91 \n",
       "Q 1263 -91 848 271 \n",
       "Q 434 634 434 1313 \n",
       "Q 434 1759 690 2068 \n",
       "Q 947 2378 1403 2484 \n",
       "z\n",
       "M 1172 3481 \n",
       "Q 1172 3119 1398 2916 \n",
       "Q 1625 2713 2034 2713 \n",
       "Q 2441 2713 2670 2916 \n",
       "Q 2900 3119 2900 3481 \n",
       "Q 2900 3844 2670 4047 \n",
       "Q 2441 4250 2034 4250 \n",
       "Q 1625 4250 1398 4047 \n",
       "Q 1172 3844 1172 3481 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mc38b9e0f66\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 10 -->\n",
       "      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 30.103125 144.879346 \n",
       "L 225.403125 144.879346 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <defs>\n",
       "       <path id=\"m3717093f81\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m3717093f81\" x=\"30.103125\" y=\"144.879346\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.0 -->\n",
       "      <g transform=\"translate(7.2 148.678565) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 30.103125 117.004974 \n",
       "L 225.403125 117.004974 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m3717093f81\" x=\"30.103125\" y=\"117.004974\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(7.2 120.804193) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 30.103125 89.130601 \n",
       "L 225.403125 89.130601 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m3717093f81\" x=\"30.103125\" y=\"89.130601\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(7.2 92.92982) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 30.103125 61.256229 \n",
       "L 225.403125 61.256229 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m3717093f81\" x=\"30.103125\" y=\"61.256229\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(7.2 65.055448) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <path d=\"M 30.103125 33.381857 \n",
       "L 225.403125 33.381857 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m3717093f81\" x=\"30.103125\" y=\"33.381857\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 2.0 -->\n",
       "      <g transform=\"translate(7.2 37.181075) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_21\">\n",
       "    <path d=\"M 12.743125 13.5 \n",
       "L 17.083125 14.653838 \n",
       "L 21.423125 15.15078 \n",
       "L 25.763125 15.426571 \n",
       "L 30.103125 15.596384 \n",
       "L 34.443125 16.35792 \n",
       "L 38.783125 16.369928 \n",
       "L 43.123125 16.369826 \n",
       "L 47.463125 16.378907 \n",
       "L 51.803125 16.393132 \n",
       "L 56.143125 16.416337 \n",
       "L 60.483125 16.493241 \n",
       "L 64.823125 16.619245 \n",
       "L 69.163125 18.45343 \n",
       "L 73.503125 26.864783 \n",
       "L 77.843125 78.518719 \n",
       "L 82.183125 82.798615 \n",
       "L 86.523125 85.403892 \n",
       "L 90.863125 87.459246 \n",
       "L 95.203125 89.00085 \n",
       "L 99.543125 97.091235 \n",
       "L 103.883125 99.256209 \n",
       "L 108.223125 100.127332 \n",
       "L 112.563125 101.136712 \n",
       "L 116.903125 101.890183 \n",
       "L 121.243125 105.691737 \n",
       "L 125.583125 105.901376 \n",
       "L 129.923125 106.797406 \n",
       "L 134.263125 106.893269 \n",
       "L 138.603125 107.370667 \n",
       "L 142.943125 108.155869 \n",
       "L 147.283125 108.89484 \n",
       "L 151.623125 109.629458 \n",
       "L 155.963125 110.060997 \n",
       "L 160.303125 110.354308 \n",
       "L 164.643125 111.515031 \n",
       "L 168.983125 112.173291 \n",
       "L 173.323125 112.564321 \n",
       "L 177.663125 112.689239 \n",
       "L 182.003125 113.09256 \n",
       "L 186.343125 113.359131 \n",
       "L 190.683125 114.161298 \n",
       "L 195.023125 114.807785 \n",
       "L 199.363125 115.17374 \n",
       "L 203.703125 115.396117 \n",
       "L 208.043125 117.058268 \n",
       "L 212.383125 116.983422 \n",
       "L 216.723125 117.333071 \n",
       "L 221.063125 117.188236 \n",
       "L 225.403125 117.093512 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_22\">\n",
       "    <path d=\"M 12.743125 139.300765 \n",
       "L 17.083125 139.402699 \n",
       "L 21.423125 139.38571 \n",
       "L 25.763125 139.281073 \n",
       "L 30.103125 139.255227 \n",
       "L 34.443125 139.5 \n",
       "L 38.783125 139.391116 \n",
       "L 43.123125 139.368721 \n",
       "L 47.463125 139.312348 \n",
       "L 51.803125 139.252439 \n",
       "L 56.143125 138.860595 \n",
       "L 60.483125 138.788777 \n",
       "L 64.823125 138.457491 \n",
       "L 69.163125 137.227331 \n",
       "L 73.503125 134.157933 \n",
       "L 77.843125 116.031965 \n",
       "L 82.183125 114.512219 \n",
       "L 86.523125 113.253486 \n",
       "L 90.863125 112.258083 \n",
       "L 95.203125 111.66889 \n",
       "L 99.543125 108.257165 \n",
       "L 103.883125 107.152106 \n",
       "L 108.223125 106.776031 \n",
       "L 112.563125 106.363274 \n",
       "L 116.903125 106.022471 \n",
       "L 121.243125 104.559734 \n",
       "L 125.583125 104.536567 \n",
       "L 129.923125 104.125741 \n",
       "L 134.263125 104.048904 \n",
       "L 138.603125 103.824112 \n",
       "L 142.943125 103.429191 \n",
       "L 147.283125 103.167405 \n",
       "L 151.623125 102.788241 \n",
       "L 155.963125 102.701751 \n",
       "L 160.303125 102.560474 \n",
       "L 164.643125 101.997478 \n",
       "L 168.983125 101.731059 \n",
       "L 173.323125 101.56503 \n",
       "L 177.663125 101.487807 \n",
       "L 182.003125 101.339577 \n",
       "L 186.343125 101.432207 \n",
       "L 190.683125 100.957286 \n",
       "L 195.023125 100.650711 \n",
       "L 199.363125 100.500899 \n",
       "L 203.703125 100.416006 \n",
       "L 208.043125 99.713225 \n",
       "L 212.383125 99.743342 \n",
       "L 216.723125 99.517079 \n",
       "L 221.063125 99.589283 \n",
       "L 225.403125 99.617869 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_23\">\n",
       "    <path d=\"M 30.103125 139.304472 \n",
       "L 51.803125 139.304472 \n",
       "L 73.503125 119.747812 \n",
       "L 95.203125 109.339521 \n",
       "L 116.903125 108.536739 \n",
       "L 138.603125 103.396705 \n",
       "L 160.303125 104.327709 \n",
       "L 182.003125 101.579296 \n",
       "L 203.703125 101.696368 \n",
       "L 225.403125 100.180003 \n",
       "\" clip-path=\"url(#p9d99a5132b)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 30.103125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 30.103125 7.2 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 140.634375 59.234375 \n",
       "L 218.403125 59.234375 \n",
       "Q 220.403125 59.234375 220.403125 57.234375 \n",
       "L 220.403125 14.2 \n",
       "Q 220.403125 12.2 218.403125 12.2 \n",
       "L 140.634375 12.2 \n",
       "Q 138.634375 12.2 138.634375 14.2 \n",
       "L 138.634375 57.234375 \n",
       "Q 138.634375 59.234375 140.634375 59.234375 \n",
       "z\n",
       "\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_24\">\n",
       "     <path d=\"M 142.634375 20.298438 \n",
       "L 152.634375 20.298438 \n",
       "L 162.634375 20.298438 \n",
       "\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- train loss -->\n",
       "     <g transform=\"translate(170.634375 23.798438) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_25\">\n",
       "     <path d=\"M 142.634375 34.976563 \n",
       "L 152.634375 34.976563 \n",
       "L 162.634375 34.976563 \n",
       "\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- train acc -->\n",
       "     <g transform=\"translate(170.634375 38.476563) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_26\">\n",
       "     <path d=\"M 142.634375 49.654688 \n",
       "L 152.634375 49.654688 \n",
       "L 162.634375 49.654688 \n",
       "\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- test acc -->\n",
       "     <g transform=\"translate(170.634375 53.154688) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p9d99a5132b\">\n",
       "   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 创建一个LeNet模型，包括卷积层、池化层和全连接层\n",
    "net = nn.Sequential(\n",
    "    # 添加第一个卷积层，输入通道数为1，输出通道数为6，卷积核大小为5x5，padding为2\n",
    "    nn.Conv2d(1, 6, kernel_size=5, padding=2), \n",
    "    # 使用Sigmoid激活函数\n",
    "    nn.Sigmoid(),  \n",
    "    # 添加第一个平均池化层，使用2x2的池化窗口，步长为2\n",
    "    nn.AvgPool2d(kernel_size=2, stride=2), \n",
    "    # 添加第二个卷积层，输入通道数为6，输出通道数为16，卷积核大小为5x5\n",
    "    nn.Conv2d(6, 16, kernel_size=5), \n",
    "    # 使用Sigmoid激活函数\n",
    "    nn.Sigmoid(),  \n",
    "    # 添加第二个平均池化层，使用2x2的池化窗口，步长为2\n",
    "    nn.AvgPool2d(kernel_size=2, stride=2), \n",
    "    # 展平层，将多维输入展平为一维\n",
    "    nn.Flatten(),  \n",
    "    # 添加第一个全连接层，输入大小为16*5*5，输出大小为120\n",
    "    nn.Linear(16 * 5 * 5, 120), \n",
    "    # 使用Sigmoid激活函数\n",
    "    nn.Sigmoid(),  \n",
    "    # 添加第二个全连接层，输入大小为120，输出大小为84\n",
    "    nn.Linear(120, 84), \n",
    "    # 使用Sigmoid激活函数\n",
    "    nn.Sigmoid(),  \n",
    "    # 添加第三个全连接层，输入大小为84，输出大小为10（对应10个类别的分类）\n",
    "    nn.Linear(84, 10))  \n",
    "\n",
    "# 创建一个随机输入张量\n",
    "X = torch.rand(size=(1, 1, 28, 28), dtype=torch.float32)\n",
    "\n",
    "# 遍历LeNet模型的各层并输出各层的输出形状\n",
    "for layer in net:\n",
    "    X = layer(X)\n",
    "    print(layer.__class__.__name__,'output shape: \\t',X.shape)\n",
    "\n",
    "# 加载Fashion MNIST数据集\n",
    "batch_size = 256\n",
    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size=batch_size)\n",
    "\n",
    "# 定义用于在GPU上评估模型精度的函数\n",
    "def evaluate_accuracy_gpu(net, data_iter, device=None): #@save\n",
    "    \"\"\"使用GPU计算模型在数据集上的精度\"\"\"\n",
    "    if isinstance(net, nn.Module):\n",
    "        net.eval()  # 设置为评估模式\n",
    "        if not device:\n",
    "            device = next(iter(net.parameters())).device\n",
    "    # 正确预测的数量，总预测的数量\n",
    "    metric = d2l.Accumulator(2)\n",
    "    with torch.no_grad():\n",
    "        for X, y in data_iter:\n",
    "            if isinstance(X, list):\n",
    "                # BERT微调所需的（之后将介绍）\n",
    "                X = [x.to(device) for x in X]\n",
    "            else:\n",
    "                X = X.to(device)\n",
    "            y = y.to(device)\n",
    "            metric.add(d2l.accuracy(net(X), y), y.numel())\n",
    "    return metric[0] / metric[1]\n",
    "\n",
    "# 定义用于训练模型的函数@save\n",
    "def train_ch6(net, train_iter, test_iter, num_epochs, lr, device):\n",
    "    \"\"\"用GPU训练模型(在第六章定义)\"\"\"\n",
    "    # 定义初始化权重的函数，使用Xavier初始化\n",
    "    def init_weights(m):\n",
    "        if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
    "            nn.init.xavier_uniform_(m.weight)\n",
    "    # 对模型的所有层应用初始化权重的函数\n",
    "    net.apply(init_weights)\n",
    "    # 打印在哪个设备上进行训练（CPU或GPU）\n",
    "    print('training on', device)\n",
    "    # 将模型移动到指定的设备（CPU或GPU）\n",
    "    net.to(device)\n",
    "    # 定义优化器，使用随机梯度下降（SGD）优化器\n",
    "    optimizer = torch.optim.SGD(net.parameters(), lr=lr)\n",
    "    # 定义损失函数，使用交叉熵损失\n",
    "    loss = nn.CrossEntropyLoss()\n",
    "    # 创建用于绘制训练过程的动画\n",
    "    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n",
    "                            legend=['train loss', 'train acc', 'test acc'])\n",
    "    # 创建计时器和获取每个epoch的批量数\n",
    "    timer, num_batches = d2l.Timer(), len(train_iter)\n",
    "    for epoch in range(num_epochs):\n",
    "        # 训练损失之和，训练准确率之和，样本数\n",
    "        metric = d2l.Accumulator(3)\n",
    "        # 设置模型为训练模式\n",
    "        net.train()\n",
    "        for i, (X, y) in enumerate(train_iter):\n",
    "            timer.start()\n",
    "            optimizer.zero_grad()\n",
    "            # 将输入数据和标签移动到指定的设备上\n",
    "            X, y = X.to(device), y.to(device)\n",
    "            # 前向传播\n",
    "            y_hat = net(X)\n",
    "            # 计算损失\n",
    "            l = loss(y_hat, y)\n",
    "            # 反向传播\n",
    "            l.backward()\n",
    "            # 更新模型参数\n",
    "            optimizer.step()\n",
    "            with torch.no_grad():\n",
    "                # 记录训练损失、训练准确率和样本数\n",
    "                metric.add(l * X.shape[0], d2l.accuracy(y_hat, y), X.shape[0])\n",
    "            timer.stop()\n",
    "            train_l = metric[0] / metric[2]\n",
    "            train_acc = metric[1] / metric[2]\n",
    "            # 每完成一定批量数的训练，更新动画\n",
    "            if (i + 1) % (num_batches // 5) == 0 or i == num_batches - 1:\n",
    "                animator.add(epoch + (i + 1) / num_batches,\n",
    "                            (train_l, train_acc, None))\n",
    "        # 计算并记录测试集上的准确率\n",
    "        test_acc = evaluate_accuracy_gpu(net, test_iter)\n",
    "        animator.add(epoch + 1, (None, None, test_acc))\n",
    "    # 打印最终的训练损失、训练准确率和测试准确率\n",
    "    print(f'loss {train_l:.3f}, train acc {train_acc:.3f}, '\n",
    "        f'test acc {test_acc:.3f}')\n",
    "    # 打印训练速度（样本/秒）\n",
    "    print(f'{metric[2] * num_epochs / timer.sum():.1f} examples/sec '\n",
    "        f'on {str(device)}')\n",
    "\n",
    "# 设置学习率和训练周期，并开始训练LeNet模型\n",
    "lr, num_epochs = 0.9, 10\n",
    "\n",
    "train_ch6(net, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pNRiATeIuQMX"
   },
   "source": [
    "&emsp;&emsp; 代码验证如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "f4384f90",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.437, train acc 0.837, test acc 0.810\n",
      "147062.3 examples/sec on cuda:0\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-03-01T02:08:02.974862</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 238.965625 183.35625 \n",
       "L 238.965625 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "L 30.103125 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 51.803125 145.8 \n",
       "L 51.803125 7.2 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"m8765e965c0\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m8765e965c0\" x=\"51.803125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(48.621875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 95.203125 145.8 \n",
       "L 95.203125 7.2 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m8765e965c0\" x=\"95.203125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 4 -->\n",
       "      <g transform=\"translate(92.021875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 138.603125 145.8 \n",
       "L 138.603125 7.2 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m8765e965c0\" x=\"138.603125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 6 -->\n",
       "      <g transform=\"translate(135.421875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
       "Q 1688 2584 1439 2293 \n",
       "Q 1191 2003 1191 1497 \n",
       "Q 1191 994 1439 701 \n",
       "Q 1688 409 2113 409 \n",
       "Q 2538 409 2786 701 \n",
       "Q 3034 994 3034 1497 \n",
       "Q 3034 2003 2786 2293 \n",
       "Q 2538 2584 2113 2584 \n",
       "z\n",
       "M 3366 4563 \n",
       "L 3366 3988 \n",
       "Q 3128 4100 2886 4159 \n",
       "Q 2644 4219 2406 4219 \n",
       "Q 1781 4219 1451 3797 \n",
       "Q 1122 3375 1075 2522 \n",
       "Q 1259 2794 1537 2939 \n",
       "Q 1816 3084 2150 3084 \n",
       "Q 2853 3084 3261 2657 \n",
       "Q 3669 2231 3669 1497 \n",
       "Q 3669 778 3244 343 \n",
       "Q 2819 -91 2113 -91 \n",
       "Q 1303 -91 875 529 \n",
       "Q 447 1150 447 2328 \n",
       "Q 447 3434 972 4092 \n",
       "Q 1497 4750 2381 4750 \n",
       "Q 2619 4750 2861 4703 \n",
       "Q 3103 4656 3366 4563 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 182.003125 145.8 \n",
       "L 182.003125 7.2 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m8765e965c0\" x=\"182.003125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 8 -->\n",
       "      <g transform=\"translate(178.821875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
       "Q 1584 2216 1326 1975 \n",
       "Q 1069 1734 1069 1313 \n",
       "Q 1069 891 1326 650 \n",
       "Q 1584 409 2034 409 \n",
       "Q 2484 409 2743 651 \n",
       "Q 3003 894 3003 1313 \n",
       "Q 3003 1734 2745 1975 \n",
       "Q 2488 2216 2034 2216 \n",
       "z\n",
       "M 1403 2484 \n",
       "Q 997 2584 770 2862 \n",
       "Q 544 3141 544 3541 \n",
       "Q 544 4100 942 4425 \n",
       "Q 1341 4750 2034 4750 \n",
       "Q 2731 4750 3128 4425 \n",
       "Q 3525 4100 3525 3541 \n",
       "Q 3525 3141 3298 2862 \n",
       "Q 3072 2584 2669 2484 \n",
       "Q 3125 2378 3379 2068 \n",
       "Q 3634 1759 3634 1313 \n",
       "Q 3634 634 3220 271 \n",
       "Q 2806 -91 2034 -91 \n",
       "Q 1263 -91 848 271 \n",
       "Q 434 634 434 1313 \n",
       "Q 434 1759 690 2068 \n",
       "Q 947 2378 1403 2484 \n",
       "z\n",
       "M 1172 3481 \n",
       "Q 1172 3119 1398 2916 \n",
       "Q 1625 2713 2034 2713 \n",
       "Q 2441 2713 2670 2916 \n",
       "Q 2900 3119 2900 3481 \n",
       "Q 2900 3844 2670 4047 \n",
       "Q 2441 4250 2034 4250 \n",
       "Q 1625 4250 1398 4047 \n",
       "Q 1172 3844 1172 3481 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m8765e965c0\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 10 -->\n",
       "      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 30.103125 145.068811 \n",
       "L 225.403125 145.068811 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <defs>\n",
       "       <path id=\"m56b613a6ac\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m56b613a6ac\" x=\"30.103125\" y=\"145.068811\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.0 -->\n",
       "      <g transform=\"translate(7.2 148.86803) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 30.103125 117.162132 \n",
       "L 225.403125 117.162132 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m56b613a6ac\" x=\"30.103125\" y=\"117.162132\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(7.2 120.961351) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 30.103125 89.255452 \n",
       "L 225.403125 89.255452 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m56b613a6ac\" x=\"30.103125\" y=\"89.255452\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(7.2 93.054671) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 30.103125 61.348773 \n",
       "L 225.403125 61.348773 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m56b613a6ac\" x=\"30.103125\" y=\"61.348773\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(7.2 65.147991) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <path d=\"M 30.103125 33.442093 \n",
       "L 225.403125 33.442093 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m56b613a6ac\" x=\"30.103125\" y=\"33.442093\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 2.0 -->\n",
       "      <g transform=\"translate(7.2 37.241312) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_21\">\n",
       "    <path d=\"M 12.743125 13.5 \n",
       "L 17.083125 14.727795 \n",
       "L 21.423125 15.199072 \n",
       "L 25.763125 15.513616 \n",
       "L 30.103125 15.756541 \n",
       "L 34.443125 24.507242 \n",
       "L 38.783125 42.341129 \n",
       "L 43.123125 54.633881 \n",
       "L 47.463125 62.503833 \n",
       "L 51.803125 68.078629 \n",
       "L 56.143125 92.870872 \n",
       "L 60.483125 95.119411 \n",
       "L 64.823125 96.823244 \n",
       "L 69.163125 97.985495 \n",
       "L 73.503125 99.267264 \n",
       "L 77.843125 104.156451 \n",
       "L 82.183125 105.342885 \n",
       "L 86.523125 106.220346 \n",
       "L 90.863125 107.037785 \n",
       "L 95.203125 107.424326 \n",
       "L 99.543125 109.356738 \n",
       "L 103.883125 110.504985 \n",
       "L 108.223125 110.772654 \n",
       "L 112.563125 111.03795 \n",
       "L 116.903125 111.286048 \n",
       "L 121.243125 113.629264 \n",
       "L 125.583125 113.91939 \n",
       "L 129.923125 114.222917 \n",
       "L 134.263125 114.191619 \n",
       "L 138.603125 114.298352 \n",
       "L 142.943125 115.40583 \n",
       "L 147.283125 115.406877 \n",
       "L 151.623125 115.997779 \n",
       "L 155.963125 116.284928 \n",
       "L 160.303125 116.517889 \n",
       "L 164.643125 118.093785 \n",
       "L 168.983125 118.27277 \n",
       "L 173.323125 117.849294 \n",
       "L 177.663125 117.903212 \n",
       "L 182.003125 118.044031 \n",
       "L 186.343125 119.528946 \n",
       "L 190.683125 118.900791 \n",
       "L 195.023125 119.253264 \n",
       "L 199.363125 119.24437 \n",
       "L 203.703125 119.522454 \n",
       "L 208.043125 120.202062 \n",
       "L 212.383125 120.460974 \n",
       "L 216.723125 120.826297 \n",
       "L 221.063125 120.575706 \n",
       "L 225.403125 120.655805 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_22\">\n",
       "    <path d=\"M 12.743125 139.386351 \n",
       "L 17.083125 139.5 \n",
       "L 21.423125 139.486857 \n",
       "L 25.763125 139.410704 \n",
       "L 30.103125 139.331198 \n",
       "L 34.443125 133.546173 \n",
       "L 38.783125 128.645341 \n",
       "L 43.123125 124.491346 \n",
       "L 47.463125 121.70694 \n",
       "L 51.803125 119.699779 \n",
       "L 56.143125 110.282876 \n",
       "L 60.483125 109.209006 \n",
       "L 64.823125 108.585095 \n",
       "L 69.163125 108.099187 \n",
       "L 73.503125 107.526886 \n",
       "L 77.843125 105.337975 \n",
       "L 82.183125 104.899614 \n",
       "L 86.523125 104.496816 \n",
       "L 90.863125 104.185247 \n",
       "L 95.203125 104.030179 \n",
       "L 99.543125 103.408258 \n",
       "L 103.883125 102.851609 \n",
       "L 108.223125 102.681522 \n",
       "L 112.563125 102.586041 \n",
       "L 116.903125 102.465544 \n",
       "L 121.243125 101.557399 \n",
       "L 125.583125 101.402002 \n",
       "L 129.923125 101.226502 \n",
       "L 134.263125 101.2582 \n",
       "L 138.603125 101.222767 \n",
       "L 142.943125 100.713148 \n",
       "L 147.283125 100.845352 \n",
       "L 151.623125 100.52296 \n",
       "L 155.963125 100.351326 \n",
       "L 160.303125 100.285102 \n",
       "L 164.643125 99.613766 \n",
       "L 168.983125 99.479242 \n",
       "L 173.323125 99.658607 \n",
       "L 177.663125 99.601009 \n",
       "L 182.003125 99.563249 \n",
       "L 186.343125 98.82518 \n",
       "L 190.683125 99.219473 \n",
       "L 195.023125 99.041654 \n",
       "L 199.363125 99.006091 \n",
       "L 203.703125 98.915814 \n",
       "L 208.043125 98.546855 \n",
       "L 212.383125 98.356667 \n",
       "L 216.723125 98.277808 \n",
       "L 221.063125 98.357826 \n",
       "L 225.403125 98.340007 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_23\">\n",
       "    <path d=\"M 30.103125 139.487475 \n",
       "L 51.803125 111.664516 \n",
       "L 73.503125 110.096161 \n",
       "L 95.203125 103.253443 \n",
       "L 116.903125 104.442267 \n",
       "L 138.603125 102.11485 \n",
       "L 160.303125 102.421824 \n",
       "L 182.003125 100.624633 \n",
       "L 203.703125 98.654422 \n",
       "L 225.403125 99.882316 \n",
       "\" clip-path=\"url(#pd701544854)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 30.103125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 30.103125 7.2 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 140.634375 59.234375 \n",
       "L 218.403125 59.234375 \n",
       "Q 220.403125 59.234375 220.403125 57.234375 \n",
       "L 220.403125 14.2 \n",
       "Q 220.403125 12.2 218.403125 12.2 \n",
       "L 140.634375 12.2 \n",
       "Q 138.634375 12.2 138.634375 14.2 \n",
       "L 138.634375 57.234375 \n",
       "Q 138.634375 59.234375 140.634375 59.234375 \n",
       "z\n",
       "\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_24\">\n",
       "     <path d=\"M 142.634375 20.298438 \n",
       "L 152.634375 20.298438 \n",
       "L 162.634375 20.298438 \n",
       "\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- train loss -->\n",
       "     <g transform=\"translate(170.634375 23.798438) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_25\">\n",
       "     <path d=\"M 142.634375 34.976563 \n",
       "L 152.634375 34.976563 \n",
       "L 162.634375 34.976563 \n",
       "\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- train acc -->\n",
       "     <g transform=\"translate(170.634375 38.476563) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_26\">\n",
       "     <path d=\"M 142.634375 49.654688 \n",
       "L 152.634375 49.654688 \n",
       "L 162.634375 49.654688 \n",
       "\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- test acc -->\n",
       "     <g transform=\"translate(170.634375 53.154688) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pd701544854\">\n",
       "   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义一个包含最大池化层的LeNet模型\n",
    "net_maxpool = nn.Sequential(\n",
    "    # 添加第一个卷积层，输入通道数为1，输出通道数为6，卷积核大小为5x5，padding为2\n",
    "    nn.Conv2d(1, 6, kernel_size=5, padding=2), \n",
    "    # 使用Sigmoid激活函数\n",
    "    nn.Sigmoid(),  \n",
    "    # 添加第一个最大池化层，使用2x2的池化窗口，步长为2\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),  \n",
    "    # 添加第二个卷积层，输入通道数为6，输出通道数为16，卷积核大小为5x5\n",
    "    nn.Conv2d(6, 16, kernel_size=5), \n",
    "    # 使用Sigmoid激活函数\n",
    "    nn.Sigmoid(),  \n",
    "    # 添加第二个最大池化层，使用2x2的池化窗口，步长为2\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),  \n",
    "    # 展平层，将多维输入展平为一维\n",
    "    nn.Flatten(),  \n",
    "    # 添加第一个全连接层，输入大小为16*5*5，输出大小为120\n",
    "    nn.Linear(16 * 5 * 5, 120), \n",
    "    # 使用Sigmoid激活函数\n",
    "    nn.Sigmoid(),  \n",
    "    # 添加第二个全连接层，输入大小为120，输出大小为84\n",
    "    nn.Linear(120, 84), \n",
    "    # 使用Sigmoid激活函数\n",
    "    nn.Sigmoid(),  \n",
    "     # 添加第三个全连接层，输入大小为84，输出大小为10（对应10个类别的分类）\n",
    "    nn.Linear(84, 10))  \n",
    "\n",
    "# 创建一个随机输入张量\n",
    "X = torch.rand(size=(1, 1, 28, 28), dtype=torch.float32)\n",
    "\n",
    "# 遍历LeNet模型的各层并输出各层的输出形状\n",
    "for layer in net_maxpool:\n",
    "    X = layer(X)\n",
    "    print(layer.__class__.__name__,'output shape: \\t',X.shape)\n",
    "\n",
    "# 加载Fashion MNIST数据集\n",
    "batch_size = 256\n",
    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size=batch_size)\n",
    "\n",
    "# 定义用于在GPU上评估模型精度的函数\n",
    "def evaluate_accuracy_gpu(net, data_iter, device=None): #@save\n",
    "    \"\"\"使用GPU计算模型在数据集上的精度\"\"\"\n",
    "    if isinstance(net, nn.Module):\n",
    "        net.eval()  # 设置为评估模式\n",
    "        if not device:\n",
    "            device = next(iter(net.parameters())).device\n",
    "    # 正确预测的数量，总预测的数量\n",
    "    metric = d2l.Accumulator(2)\n",
    "    with torch.no_grad():\n",
    "        for X, y in data_iter:\n",
    "            if isinstance(X, list):\n",
    "                # BERT微调所需的（之后将介绍）\n",
    "                X = [x.to(device) for x in X]\n",
    "            else:\n",
    "                X = X.to(device)\n",
    "            y = y.to(device)\n",
    "            metric.add(d2l.accuracy(net(X), y), y.numel())\n",
    "    return metric[0] / metric[1]\n",
    "\n",
    "# 定义用于训练模型的函数@save\n",
    "def train_ch6(net, train_iter, test_iter, num_epochs, lr, device):\n",
    "    \"\"\"用GPU训练模型(在第六章定义)\"\"\"\n",
    "    # 定义初始化权重的函数，使用Xavier初始化\n",
    "    def init_weights(m):\n",
    "        if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
    "            nn.init.xavier_uniform_(m.weight)\n",
    "    # 对模型的所有层应用初始化权重的函数\n",
    "    net.apply(init_weights)\n",
    "    # 打印在哪个设备上进行训练（CPU或GPU）\n",
    "    print('training on', device)\n",
    "    # 将模型移动到指定的设备（CPU或GPU）\n",
    "    net.to(device)\n",
    "    # 定义优化器，使用随机梯度下降（SGD）优化器\n",
    "    optimizer = torch.optim.SGD(net.parameters(), lr=lr)\n",
    "    # 定义损失函数，使用交叉熵损失\n",
    "    loss = nn.CrossEntropyLoss()\n",
    "    # 创建用于绘制训练过程的动画\n",
    "    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n",
    "                            legend=['train loss', 'train acc', 'test acc'])\n",
    "    # 创建计时器和获取每个epoch的批量数\n",
    "    timer, num_batches = d2l.Timer(), len(train_iter)\n",
    "    for epoch in range(num_epochs):\n",
    "        # 训练损失之和，训练准确率之和，样本数\n",
    "        metric = d2l.Accumulator(3)\n",
    "        # 设置模型为训练模式\n",
    "        net.train()\n",
    "        for i, (X, y) in enumerate(train_iter):\n",
    "            timer.start()\n",
    "            optimizer.zero_grad()\n",
    "            # 将输入数据和标签移动到指定的设备上\n",
    "            X, y = X.to(device), y.to(device)\n",
    "            # 前向传播\n",
    "            y_hat = net(X)\n",
    "            # 计算损失\n",
    "            l = loss(y_hat, y)\n",
    "            # 反向传播\n",
    "            l.backward()\n",
    "            # 更新模型参数\n",
    "            optimizer.step()\n",
    "            with torch.no_grad():\n",
    "                # 记录训练损失、训练准确率和样本数\n",
    "                metric.add(l * X.shape[0], d2l.accuracy(y_hat, y), X.shape[0])\n",
    "            timer.stop()\n",
    "            train_l = metric[0] / metric[2]\n",
    "            train_acc = metric[1] / metric[2]\n",
    "            # 每完成一定批量数的训练，更新动画\n",
    "            if (i + 1) % (num_batches // 5) == 0 or i == num_batches - 1:\n",
    "                animator.add(epoch + (i + 1) / num_batches,\n",
    "                            (train_l, train_acc, None))\n",
    "        # 计算并记录测试集上的准确率\n",
    "        test_acc = evaluate_accuracy_gpu(net, test_iter)\n",
    "        animator.add(epoch + 1, (None, None, test_acc))\n",
    "    # 打印最终的训练损失、训练准确率和测试准确率\n",
    "    print(f'loss {train_l:.3f}, train acc {train_acc:.3f}, '\n",
    "        f'test acc {test_acc:.3f}')\n",
    "    # 打印训练速度（样本/秒）\n",
    "    print(f'{metric[2] * num_epochs / timer.sum():.1f} examples/sec '\n",
    "        f'on {str(device)}')\n",
    "\n",
    "# 设置学习率和训练周期\n",
    "lr, num_epochs = 0.9, 10\n",
    "\n",
    "# 调用train_ch6函数来在GPU上训练模型，使用你之前定义的函数\n",
    "train_ch6(net_maxpool, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "q-RxShsf46LS"
   },
   "source": [
    "&emsp;&emsp; 可见用最大汇聚层替换平均汇聚层后，在训练集和测试集上均得到了更好的结果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "etARpxbiH6Gt"
   },
   "source": [
    "### 练习 6.6.2\n",
    "\n",
    "尝试构建一个基于`LeNet`的更复杂的网络，以提高其准确性。\n",
    "1. 调整卷积窗口大小。\n",
    "2. 调整输出通道的数量。\n",
    "3. 调整激活函数（如`ReLU`）。\n",
    "4. 调整卷积层的数量。\n",
    "5. 调整全连接层的数量。\n",
    "6. 调整学习率和其他训练细节（例如，初始化和轮数）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xmc7aQICH6Gt"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sjvzTbAXulfz"
   },
   "source": [
    "**第1问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&emsp;&emsp;修改为`nn.Conv2d(1, 6, kernel_size = 7)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.434, train acc 0.840, test acc 0.780\n",
      "154221.3 examples/sec on cuda:0\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-03-01T02:08:31.192606</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 238.965625 183.35625 \n",
       "L 238.965625 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "L 30.103125 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 51.803125 145.8 \n",
       "L 51.803125 7.2 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"m3c19442243\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m3c19442243\" x=\"51.803125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(48.621875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 95.203125 145.8 \n",
       "L 95.203125 7.2 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m3c19442243\" x=\"95.203125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 4 -->\n",
       "      <g transform=\"translate(92.021875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 138.603125 145.8 \n",
       "L 138.603125 7.2 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m3c19442243\" x=\"138.603125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 6 -->\n",
       "      <g transform=\"translate(135.421875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
       "Q 1688 2584 1439 2293 \n",
       "Q 1191 2003 1191 1497 \n",
       "Q 1191 994 1439 701 \n",
       "Q 1688 409 2113 409 \n",
       "Q 2538 409 2786 701 \n",
       "Q 3034 994 3034 1497 \n",
       "Q 3034 2003 2786 2293 \n",
       "Q 2538 2584 2113 2584 \n",
       "z\n",
       "M 3366 4563 \n",
       "L 3366 3988 \n",
       "Q 3128 4100 2886 4159 \n",
       "Q 2644 4219 2406 4219 \n",
       "Q 1781 4219 1451 3797 \n",
       "Q 1122 3375 1075 2522 \n",
       "Q 1259 2794 1537 2939 \n",
       "Q 1816 3084 2150 3084 \n",
       "Q 2853 3084 3261 2657 \n",
       "Q 3669 2231 3669 1497 \n",
       "Q 3669 778 3244 343 \n",
       "Q 2819 -91 2113 -91 \n",
       "Q 1303 -91 875 529 \n",
       "Q 447 1150 447 2328 \n",
       "Q 447 3434 972 4092 \n",
       "Q 1497 4750 2381 4750 \n",
       "Q 2619 4750 2861 4703 \n",
       "Q 3103 4656 3366 4563 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 182.003125 145.8 \n",
       "L 182.003125 7.2 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m3c19442243\" x=\"182.003125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 8 -->\n",
       "      <g transform=\"translate(178.821875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
       "Q 1584 2216 1326 1975 \n",
       "Q 1069 1734 1069 1313 \n",
       "Q 1069 891 1326 650 \n",
       "Q 1584 409 2034 409 \n",
       "Q 2484 409 2743 651 \n",
       "Q 3003 894 3003 1313 \n",
       "Q 3003 1734 2745 1975 \n",
       "Q 2488 2216 2034 2216 \n",
       "z\n",
       "M 1403 2484 \n",
       "Q 997 2584 770 2862 \n",
       "Q 544 3141 544 3541 \n",
       "Q 544 4100 942 4425 \n",
       "Q 1341 4750 2034 4750 \n",
       "Q 2731 4750 3128 4425 \n",
       "Q 3525 4100 3525 3541 \n",
       "Q 3525 3141 3298 2862 \n",
       "Q 3072 2584 2669 2484 \n",
       "Q 3125 2378 3379 2068 \n",
       "Q 3634 1759 3634 1313 \n",
       "Q 3634 634 3220 271 \n",
       "Q 2806 -91 2034 -91 \n",
       "Q 1263 -91 848 271 \n",
       "Q 434 634 434 1313 \n",
       "Q 434 1759 690 2068 \n",
       "Q 947 2378 1403 2484 \n",
       "z\n",
       "M 1172 3481 \n",
       "Q 1172 3119 1398 2916 \n",
       "Q 1625 2713 2034 2713 \n",
       "Q 2441 2713 2670 2916 \n",
       "Q 2900 3119 2900 3481 \n",
       "Q 2900 3844 2670 4047 \n",
       "Q 2441 4250 2034 4250 \n",
       "Q 1625 4250 1398 4047 \n",
       "Q 1172 3844 1172 3481 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m3c19442243\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 10 -->\n",
       "      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 30.103125 145.062947 \n",
       "L 225.403125 145.062947 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <defs>\n",
       "       <path id=\"m700a064119\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m700a064119\" x=\"30.103125\" y=\"145.062947\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.0 -->\n",
       "      <g transform=\"translate(7.2 148.862166) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 30.103125 117.366375 \n",
       "L 225.403125 117.366375 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m700a064119\" x=\"30.103125\" y=\"117.366375\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(7.2 121.165594) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 30.103125 89.669804 \n",
       "L 225.403125 89.669804 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m700a064119\" x=\"30.103125\" y=\"89.669804\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(7.2 93.469022) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 30.103125 61.973232 \n",
       "L 225.403125 61.973232 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m700a064119\" x=\"30.103125\" y=\"61.973232\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(7.2 65.77245) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <path d=\"M 30.103125 34.27666 \n",
       "L 225.403125 34.27666 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m700a064119\" x=\"30.103125\" y=\"34.27666\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 2.0 -->\n",
       "      <g transform=\"translate(7.2 38.075879) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_21\">\n",
       "    <path d=\"M 12.743125 13.5 \n",
       "L 17.083125 15.264818 \n",
       "L 21.423125 15.899156 \n",
       "L 25.763125 16.28043 \n",
       "L 30.103125 16.651314 \n",
       "L 34.443125 30.971039 \n",
       "L 38.783125 45.610639 \n",
       "L 43.123125 55.734637 \n",
       "L 47.463125 62.942339 \n",
       "L 51.803125 68.204226 \n",
       "L 56.143125 91.244312 \n",
       "L 60.483125 92.390874 \n",
       "L 64.823125 93.562466 \n",
       "L 69.163125 94.366764 \n",
       "L 73.503125 95.41084 \n",
       "L 77.843125 103.422395 \n",
       "L 82.183125 104.078819 \n",
       "L 86.523125 104.444944 \n",
       "L 90.863125 104.575793 \n",
       "L 95.203125 105.235006 \n",
       "L 99.543125 108.640359 \n",
       "L 103.883125 109.218018 \n",
       "L 108.223125 109.202378 \n",
       "L 112.563125 109.789573 \n",
       "L 116.903125 110.149577 \n",
       "L 121.243125 111.193216 \n",
       "L 125.583125 111.855984 \n",
       "L 129.923125 112.463628 \n",
       "L 134.263125 113.137546 \n",
       "L 138.603125 113.509332 \n",
       "L 142.943125 115.074203 \n",
       "L 147.283125 115.28206 \n",
       "L 151.623125 115.614744 \n",
       "L 155.963125 116.057445 \n",
       "L 160.303125 116.466749 \n",
       "L 164.643125 117.235709 \n",
       "L 168.983125 117.367716 \n",
       "L 173.323125 117.472721 \n",
       "L 177.663125 117.659928 \n",
       "L 182.003125 118.060743 \n",
       "L 186.343125 119.407099 \n",
       "L 190.683125 119.212652 \n",
       "L 195.023125 119.498365 \n",
       "L 199.363125 119.500118 \n",
       "L 203.703125 119.62992 \n",
       "L 208.043125 120.79696 \n",
       "L 212.383125 121.23137 \n",
       "L 216.723125 120.868971 \n",
       "L 221.063125 121.153845 \n",
       "L 225.403125 121.018008 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_22\">\n",
       "    <path d=\"M 12.743125 139.400251 \n",
       "L 17.083125 139.425572 \n",
       "L 21.423125 139.5 \n",
       "L 25.763125 139.418666 \n",
       "L 30.103125 139.187581 \n",
       "L 34.443125 133.783592 \n",
       "L 38.783125 129.426078 \n",
       "L 43.123125 125.777552 \n",
       "L 47.463125 123.083167 \n",
       "L 51.803125 120.969699 \n",
       "L 56.143125 111.386015 \n",
       "L 60.483125 110.944049 \n",
       "L 64.823125 110.460648 \n",
       "L 69.163125 109.968039 \n",
       "L 73.503125 109.434077 \n",
       "L 77.843125 105.571393 \n",
       "L 82.183125 105.256031 \n",
       "L 86.523125 105.130961 \n",
       "L 90.863125 105.1582 \n",
       "L 95.203125 104.983238 \n",
       "L 99.543125 103.45824 \n",
       "L 103.883125 103.405296 \n",
       "L 108.223125 103.468982 \n",
       "L 112.563125 103.261427 \n",
       "L 116.903125 103.209734 \n",
       "L 121.243125 102.781479 \n",
       "L 125.583125 102.525967 \n",
       "L 129.923125 102.270455 \n",
       "L 134.263125 102.049471 \n",
       "L 138.603125 101.901533 \n",
       "L 142.943125 101.345087 \n",
       "L 147.283125 101.220784 \n",
       "L 151.623125 101.0796 \n",
       "L 155.963125 100.92614 \n",
       "L 160.303125 100.718889 \n",
       "L 164.643125 100.488777 \n",
       "L 168.983125 100.380587 \n",
       "L 173.323125 100.295417 \n",
       "L 177.663125 100.205642 \n",
       "L 182.003125 100.009857 \n",
       "L 186.343125 99.370049 \n",
       "L 190.683125 99.333219 \n",
       "L 195.023125 99.211985 \n",
       "L 199.363125 99.219274 \n",
       "L 203.703125 99.146647 \n",
       "L 208.043125 98.776156 \n",
       "L 212.383125 98.582796 \n",
       "L 216.723125 98.661061 \n",
       "L 221.063125 98.521796 \n",
       "L 225.403125 98.546555 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_23\">\n",
       "    <path d=\"M 30.103125 135.657192 \n",
       "L 51.803125 113.433462 \n",
       "L 73.503125 105.606411 \n",
       "L 95.203125 104.697963 \n",
       "L 116.903125 106.780746 \n",
       "L 138.603125 101.474083 \n",
       "L 160.303125 101.568251 \n",
       "L 182.003125 100.056018 \n",
       "L 203.703125 99.23066 \n",
       "L 225.403125 101.839677 \n",
       "\" clip-path=\"url(#pcbc436710d)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 30.103125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 30.103125 7.2 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 140.634375 59.234375 \n",
       "L 218.403125 59.234375 \n",
       "Q 220.403125 59.234375 220.403125 57.234375 \n",
       "L 220.403125 14.2 \n",
       "Q 220.403125 12.2 218.403125 12.2 \n",
       "L 140.634375 12.2 \n",
       "Q 138.634375 12.2 138.634375 14.2 \n",
       "L 138.634375 57.234375 \n",
       "Q 138.634375 59.234375 140.634375 59.234375 \n",
       "z\n",
       "\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_24\">\n",
       "     <path d=\"M 142.634375 20.298438 \n",
       "L 152.634375 20.298438 \n",
       "L 162.634375 20.298438 \n",
       "\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- train loss -->\n",
       "     <g transform=\"translate(170.634375 23.798438) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_25\">\n",
       "     <path d=\"M 142.634375 34.976563 \n",
       "L 152.634375 34.976563 \n",
       "L 162.634375 34.976563 \n",
       "\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- train acc -->\n",
       "     <g transform=\"translate(170.634375 38.476563) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_26\">\n",
       "     <path d=\"M 142.634375 49.654688 \n",
       "L 152.634375 49.654688 \n",
       "L 162.634375 49.654688 \n",
       "\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- test acc -->\n",
       "     <g transform=\"translate(170.634375 53.154688) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pcbc436710d\">\n",
       "   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义神经网络结构\n",
    "# 创建一个序列化的神经网络模型\n",
    "net = nn.Sequential(\n",
    "    # 第一个二维卷积层。输入通道为1（例如，用于灰度图像），输出通道为6。使用7x7的卷积核，边缘填充2个像素。\n",
    "    nn.Conv2d(1, 6, kernel_size=7, padding=2), \n",
    "    # 使用Sigmoid激活函数\n",
    "    nn.Sigmoid(),\n",
    "    # 最大池化层，用于降低特征维度。使用2x2的窗口大小和步长2，减少数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第二个二维卷积层。输入通道为6（上一卷积层的输出通道数），输出通道为16。使用7x7的卷积核。\n",
    "    nn.Conv2d(6, 16, kernel_size=7), \n",
    "    # 使用Sigmoid激活函数\n",
    "    nn.Sigmoid(),\n",
    "    # 又一个最大池化层，进一步减小数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # Flatten层，将多维度的数据展平成一维，以便传递给全连接层。\n",
    "    nn.Flatten(),\n",
    "    # 全连接（线性）层。输入特征数量为16*3*3（从上一层的输出尺寸计算而来），输出特征数量为120。\n",
    "    nn.Linear(16 * 3 * 3, 120), \n",
    "    # 使用Sigmoid激活函数\n",
    "    nn.Sigmoid(),\n",
    "    # 第二个全连接（线性）层，进一步转换特征。输入特征数量为120，输出为84。\n",
    "    nn.Linear(120, 84), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第三个全连接（线性）层，最终输出特征数量为10，通常用于10类分类问题。\n",
    "    nn.Linear(84, 10))\n",
    "\n",
    "# 打印每个层的形状\n",
    "X = torch.rand(size=(1, 1, 28, 28), dtype=torch.float32)\n",
    "for layer in net:\n",
    "    X = layer(X)\n",
    "    print(layer.__class__.__name__,'output shape: \\t',X.shape)\n",
    "\n",
    "# 加载Fashion MNIST数据集\n",
    "batch_size = 256\n",
    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size=batch_size)\n",
    "\n",
    "# 定义用于在GPU上评估模型精度的函数\n",
    "def evaluate_accuracy_gpu(net, data_iter, device=None): #@save\n",
    "    \"\"\"使用GPU计算模型在数据集上的精度\"\"\"\n",
    "    if isinstance(net, nn.Module):\n",
    "        net.eval()  # 设置为评估模式\n",
    "        if not device:\n",
    "            device = next(iter(net.parameters())).device\n",
    "    # 正确预测的数量，总预测的数量\n",
    "    metric = d2l.Accumulator(2)\n",
    "    with torch.no_grad():\n",
    "        for X, y in data_iter:\n",
    "            if isinstance(X, list):\n",
    "                # BERT微调所需的（之后将介绍）\n",
    "                X = [x.to(device) for x in X]\n",
    "            else:\n",
    "                X = X.to(device)\n",
    "            y = y.to(device)\n",
    "            metric.add(d2l.accuracy(net(X), y), y.numel())\n",
    "    return metric[0] / metric[1]\n",
    "\n",
    "# 定义用于训练模型的函数@save\n",
    "def train_ch6(net, train_iter, test_iter, num_epochs, lr, device):\n",
    "    \"\"\"用GPU训练模型(在第六章定义)\"\"\"\n",
    "    # 定义初始化权重的函数，使用Xavier初始化\n",
    "    def init_weights(m):\n",
    "        if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
    "            nn.init.xavier_uniform_(m.weight)\n",
    "    # 对模型的所有层应用初始化权重的函数\n",
    "    net.apply(init_weights)\n",
    "    # 打印在哪个设备上进行训练（CPU或GPU）\n",
    "    print('training on', device)\n",
    "    # 将模型移动到指定的设备（CPU或GPU）\n",
    "    net.to(device)\n",
    "    # 定义优化器，使用随机梯度下降（SGD）优化器\n",
    "    optimizer = torch.optim.SGD(net.parameters(), lr=lr)\n",
    "    # 定义损失函数，使用交叉熵损失\n",
    "    loss = nn.CrossEntropyLoss()\n",
    "    # 创建用于绘制训练过程的动画\n",
    "    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n",
    "                            legend=['train loss', 'train acc', 'test acc'])\n",
    "    # 创建计时器和获取每个epoch的批量数\n",
    "    timer, num_batches = d2l.Timer(), len(train_iter)\n",
    "    for epoch in range(num_epochs):\n",
    "        # 训练损失之和，训练准确率之和，样本数\n",
    "        metric = d2l.Accumulator(3)\n",
    "        # 设置模型为训练模式\n",
    "        net.train()\n",
    "        for i, (X, y) in enumerate(train_iter):\n",
    "            timer.start()\n",
    "            optimizer.zero_grad()\n",
    "            # 将输入数据和标签移动到指定的设备上\n",
    "            X, y = X.to(device), y.to(device)\n",
    "            # 前向传播\n",
    "            y_hat = net(X)\n",
    "            # 计算损失\n",
    "            l = loss(y_hat, y)\n",
    "            # 反向传播\n",
    "            l.backward()\n",
    "            # 更新模型参数\n",
    "            optimizer.step()\n",
    "            with torch.no_grad():\n",
    "                # 记录训练损失、训练准确率和样本数\n",
    "                metric.add(l * X.shape[0], d2l.accuracy(y_hat, y), X.shape[0])\n",
    "            timer.stop()\n",
    "            train_l = metric[0] / metric[2]\n",
    "            train_acc = metric[1] / metric[2]\n",
    "            # 每完成一定批量数的训练，更新动画\n",
    "            if (i + 1) % (num_batches // 5) == 0 or i == num_batches - 1:\n",
    "                animator.add(epoch + (i + 1) / num_batches,\n",
    "                            (train_l, train_acc, None))\n",
    "        # 计算并记录测试集上的准确率\n",
    "        test_acc = evaluate_accuracy_gpu(net, test_iter)\n",
    "        animator.add(epoch + 1, (None, None, test_acc))\n",
    "    # 打印最终的训练损失、训练准确率和测试准确率\n",
    "    print(f'loss {train_l:.3f}, train acc {train_acc:.3f}, '\n",
    "        f'test acc {test_acc:.3f}')\n",
    "    # 打印训练速度（样本/秒）\n",
    "    print(f'{metric[2] * num_epochs / timer.sum():.1f} examples/sec '\n",
    "        f'on {str(device)}')\n",
    "\n",
    "# 设置学习率和训练周期\n",
    "lr, num_epochs = 0.9, 10\n",
    "\n",
    "# 调用train_ch6函数来在GPU上训练模型，使用你之前定义的函数\n",
    "train_ch6(net, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OnqV2DSSulx5"
   },
   "source": [
    "**第2问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Bh0S_tDful5D"
   },
   "source": [
    "&emsp;&emsp; 修改为`nn.Conv2d(1, 10, kernel_size = 5)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.408, train acc 0.849, test acc 0.826\n",
      "125885.7 examples/sec on cuda:0\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-03-01T02:09:00.397297</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 238.965625 183.35625 \n",
       "L 238.965625 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "L 30.103125 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 51.803125 145.8 \n",
       "L 51.803125 7.2 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"m6a7ff3b553\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m6a7ff3b553\" x=\"51.803125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(48.621875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 95.203125 145.8 \n",
       "L 95.203125 7.2 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m6a7ff3b553\" x=\"95.203125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 4 -->\n",
       "      <g transform=\"translate(92.021875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 138.603125 145.8 \n",
       "L 138.603125 7.2 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m6a7ff3b553\" x=\"138.603125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 6 -->\n",
       "      <g transform=\"translate(135.421875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
       "Q 1688 2584 1439 2293 \n",
       "Q 1191 2003 1191 1497 \n",
       "Q 1191 994 1439 701 \n",
       "Q 1688 409 2113 409 \n",
       "Q 2538 409 2786 701 \n",
       "Q 3034 994 3034 1497 \n",
       "Q 3034 2003 2786 2293 \n",
       "Q 2538 2584 2113 2584 \n",
       "z\n",
       "M 3366 4563 \n",
       "L 3366 3988 \n",
       "Q 3128 4100 2886 4159 \n",
       "Q 2644 4219 2406 4219 \n",
       "Q 1781 4219 1451 3797 \n",
       "Q 1122 3375 1075 2522 \n",
       "Q 1259 2794 1537 2939 \n",
       "Q 1816 3084 2150 3084 \n",
       "Q 2853 3084 3261 2657 \n",
       "Q 3669 2231 3669 1497 \n",
       "Q 3669 778 3244 343 \n",
       "Q 2819 -91 2113 -91 \n",
       "Q 1303 -91 875 529 \n",
       "Q 447 1150 447 2328 \n",
       "Q 447 3434 972 4092 \n",
       "Q 1497 4750 2381 4750 \n",
       "Q 2619 4750 2861 4703 \n",
       "Q 3103 4656 3366 4563 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 182.003125 145.8 \n",
       "L 182.003125 7.2 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m6a7ff3b553\" x=\"182.003125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 8 -->\n",
       "      <g transform=\"translate(178.821875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
       "Q 1584 2216 1326 1975 \n",
       "Q 1069 1734 1069 1313 \n",
       "Q 1069 891 1326 650 \n",
       "Q 1584 409 2034 409 \n",
       "Q 2484 409 2743 651 \n",
       "Q 3003 894 3003 1313 \n",
       "Q 3003 1734 2745 1975 \n",
       "Q 2488 2216 2034 2216 \n",
       "z\n",
       "M 1403 2484 \n",
       "Q 997 2584 770 2862 \n",
       "Q 544 3141 544 3541 \n",
       "Q 544 4100 942 4425 \n",
       "Q 1341 4750 2034 4750 \n",
       "Q 2731 4750 3128 4425 \n",
       "Q 3525 4100 3525 3541 \n",
       "Q 3525 3141 3298 2862 \n",
       "Q 3072 2584 2669 2484 \n",
       "Q 3125 2378 3379 2068 \n",
       "Q 3634 1759 3634 1313 \n",
       "Q 3634 634 3220 271 \n",
       "Q 2806 -91 2034 -91 \n",
       "Q 1263 -91 848 271 \n",
       "Q 434 634 434 1313 \n",
       "Q 434 1759 690 2068 \n",
       "Q 947 2378 1403 2484 \n",
       "z\n",
       "M 1172 3481 \n",
       "Q 1172 3119 1398 2916 \n",
       "Q 1625 2713 2034 2713 \n",
       "Q 2441 2713 2670 2916 \n",
       "Q 2900 3119 2900 3481 \n",
       "Q 2900 3844 2670 4047 \n",
       "Q 2441 4250 2034 4250 \n",
       "Q 1625 4250 1398 4047 \n",
       "Q 1172 3844 1172 3481 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m6a7ff3b553\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 10 -->\n",
       "      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 30.103125 144.968897 \n",
       "L 225.403125 144.968897 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <defs>\n",
       "       <path id=\"mb662c248c9\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#mb662c248c9\" x=\"30.103125\" y=\"144.968897\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.0 -->\n",
       "      <g transform=\"translate(7.2 148.768115) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 30.103125 117.157501 \n",
       "L 225.403125 117.157501 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mb662c248c9\" x=\"30.103125\" y=\"117.157501\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(7.2 120.956719) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 30.103125 89.346105 \n",
       "L 225.403125 89.346105 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mb662c248c9\" x=\"30.103125\" y=\"89.346105\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(7.2 93.145323) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 30.103125 61.534709 \n",
       "L 225.403125 61.534709 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mb662c248c9\" x=\"30.103125\" y=\"61.534709\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(7.2 65.333927) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <path d=\"M 30.103125 33.723313 \n",
       "L 225.403125 33.723313 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mb662c248c9\" x=\"30.103125\" y=\"33.723313\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 2.0 -->\n",
       "      <g transform=\"translate(7.2 37.522531) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_21\">\n",
       "    <path d=\"M 12.743125 13.5 \n",
       "L 17.083125 14.937317 \n",
       "L 21.423125 15.450582 \n",
       "L 25.763125 15.752328 \n",
       "L 30.103125 15.94183 \n",
       "L 34.443125 16.944229 \n",
       "L 38.783125 17.660681 \n",
       "L 43.123125 25.904418 \n",
       "L 47.463125 36.952423 \n",
       "L 51.803125 46.328628 \n",
       "L 56.143125 89.704197 \n",
       "L 60.483125 91.696319 \n",
       "L 64.823125 94.055885 \n",
       "L 69.163125 95.885092 \n",
       "L 73.503125 97.316079 \n",
       "L 77.843125 105.304196 \n",
       "L 82.183125 105.598611 \n",
       "L 86.523125 106.319774 \n",
       "L 90.863125 106.886319 \n",
       "L 95.203125 107.187394 \n",
       "L 99.543125 109.985565 \n",
       "L 103.883125 110.108204 \n",
       "L 108.223125 110.65714 \n",
       "L 112.563125 110.926843 \n",
       "L 116.903125 111.443192 \n",
       "L 121.243125 113.455722 \n",
       "L 125.583125 114.219536 \n",
       "L 129.923125 114.023692 \n",
       "L 134.263125 114.381812 \n",
       "L 138.603125 114.646521 \n",
       "L 142.943125 116.138588 \n",
       "L 147.283125 116.554052 \n",
       "L 151.623125 117.236464 \n",
       "L 155.963125 117.285054 \n",
       "L 160.303125 117.3813 \n",
       "L 164.643125 118.839942 \n",
       "L 168.983125 119.051955 \n",
       "L 173.323125 119.178958 \n",
       "L 177.663125 119.228581 \n",
       "L 182.003125 119.447549 \n",
       "L 186.343125 120.438376 \n",
       "L 190.683125 120.930173 \n",
       "L 195.023125 120.615373 \n",
       "L 199.363125 120.713283 \n",
       "L 203.703125 120.846396 \n",
       "L 208.043125 121.804704 \n",
       "L 212.383125 121.92525 \n",
       "L 216.723125 122.01543 \n",
       "L 221.063125 122.234581 \n",
       "L 225.403125 122.255774 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_22\">\n",
       "    <path d=\"M 12.743125 139.319707 \n",
       "L 17.083125 139.354378 \n",
       "L 21.423125 139.5 \n",
       "L 25.763125 139.442214 \n",
       "L 30.103125 139.378806 \n",
       "L 34.443125 138.986858 \n",
       "L 38.783125 138.110817 \n",
       "L 43.123125 134.672146 \n",
       "L 47.463125 130.829742 \n",
       "L 51.803125 127.435665 \n",
       "L 56.143125 111.711719 \n",
       "L 60.483125 110.588353 \n",
       "L 64.823125 109.603674 \n",
       "L 69.163125 108.795821 \n",
       "L 73.503125 108.314404 \n",
       "L 77.843125 105.272012 \n",
       "L 82.183125 105.114833 \n",
       "L 86.523125 104.808181 \n",
       "L 90.863125 104.506921 \n",
       "L 95.203125 104.331812 \n",
       "L 99.543125 103.210196 \n",
       "L 103.883125 103.12005 \n",
       "L 108.223125 102.942068 \n",
       "L 112.563125 102.83343 \n",
       "L 116.903125 102.633463 \n",
       "L 121.243125 101.495099 \n",
       "L 125.583125 101.250085 \n",
       "L 129.923125 101.4011 \n",
       "L 134.263125 101.23506 \n",
       "L 138.603125 101.128866 \n",
       "L 142.943125 100.575141 \n",
       "L 147.283125 100.357864 \n",
       "L 151.623125 99.983409 \n",
       "L 155.963125 99.955671 \n",
       "L 160.303125 99.872718 \n",
       "L 164.643125 99.160532 \n",
       "L 168.983125 99.132794 \n",
       "L 173.323125 99.069615 \n",
       "L 177.663125 99.026467 \n",
       "L 182.003125 98.941036 \n",
       "L 186.343125 98.522571 \n",
       "L 190.683125 98.323786 \n",
       "L 195.023125 98.464014 \n",
       "L 199.363125 98.358458 \n",
       "L 203.703125 98.30879 \n",
       "L 208.043125 97.91697 \n",
       "L 212.383125 97.875364 \n",
       "L 216.723125 97.835299 \n",
       "L 221.063125 97.76557 \n",
       "L 225.403125 97.746073 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_23\">\n",
       "    <path d=\"M 30.103125 139.406617 \n",
       "L 51.803125 113.758948 \n",
       "L 73.503125 107.696064 \n",
       "L 95.203125 103.507667 \n",
       "L 116.903125 101.761112 \n",
       "L 138.603125 102.68445 \n",
       "L 160.303125 101.321692 \n",
       "L 182.003125 99.102342 \n",
       "L 203.703125 101.277193 \n",
       "L 225.403125 99.013346 \n",
       "\" clip-path=\"url(#p97b8634564)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 30.103125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 30.103125 7.2 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 140.634375 59.234375 \n",
       "L 218.403125 59.234375 \n",
       "Q 220.403125 59.234375 220.403125 57.234375 \n",
       "L 220.403125 14.2 \n",
       "Q 220.403125 12.2 218.403125 12.2 \n",
       "L 140.634375 12.2 \n",
       "Q 138.634375 12.2 138.634375 14.2 \n",
       "L 138.634375 57.234375 \n",
       "Q 138.634375 59.234375 140.634375 59.234375 \n",
       "z\n",
       "\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_24\">\n",
       "     <path d=\"M 142.634375 20.298438 \n",
       "L 152.634375 20.298438 \n",
       "L 162.634375 20.298438 \n",
       "\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- train loss -->\n",
       "     <g transform=\"translate(170.634375 23.798438) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_25\">\n",
       "     <path d=\"M 142.634375 34.976563 \n",
       "L 152.634375 34.976563 \n",
       "L 162.634375 34.976563 \n",
       "\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- train acc -->\n",
       "     <g transform=\"translate(170.634375 38.476563) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_26\">\n",
       "     <path d=\"M 142.634375 49.654688 \n",
       "L 152.634375 49.654688 \n",
       "L 162.634375 49.654688 \n",
       "\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- test acc -->\n",
       "     <g transform=\"translate(170.634375 53.154688) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p97b8634564\">\n",
       "   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义神经网络结构\n",
    "# 创建一个序列化的神经网络模型\n",
    "net = nn.Sequential(\n",
    "    # 第一个二维卷积层。输入通道为1（例如，用于灰度图像），输出通道为10。使用5x5的卷积核，边缘填充2个像素。\n",
    "    nn.Conv2d(1, 10, kernel_size=5, padding=2), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 最大池化层，用于降低特征维度。使用2x2的窗口大小和步长2，减少数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第二个二维卷积层。输入通道为10（上一卷积层的输出通道数），输出通道为16。使用5x5的卷积核。\n",
    "    nn.Conv2d(10, 16, kernel_size=5), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 又一个最大池化层，进一步减小数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # Flatten层，将多维度的数据展平成一维，以便传递给全连接层。\n",
    "    nn.Flatten(),\n",
    "    # 全连接（线性）层。输入特征数量为16*5*5（从上一层的输出尺寸计算而来），输出特征数量为120。\n",
    "    nn.Linear(16 * 5 * 5, 120), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第二个全连接（线性）层，进一步转换特征。输入特征数量为120，输出为84。\n",
    "    nn.Linear(120, 84), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第三个全连接（线性）层，最终输出特征数量为10，通常用于10类分类问题。\n",
    "    nn.Linear(84, 10))\n",
    "\n",
    "# 打印每个层的形状\n",
    "X = torch.rand(size=(1, 1, 28, 28), dtype=torch.float32)\n",
    "for layer in net:\n",
    "    X = layer(X)\n",
    "    print(layer.__class__.__name__,'output shape: \\t',X.shape)\n",
    "\n",
    "# 加载Fashion MNIST数据集\n",
    "batch_size = 256\n",
    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size=batch_size)\n",
    "\n",
    "# 定义用于在GPU上评估模型精度的函数\n",
    "def evaluate_accuracy_gpu(net, data_iter, device=None): #@save\n",
    "    \"\"\"使用GPU计算模型在数据集上的精度\"\"\"\n",
    "    if isinstance(net, nn.Module):\n",
    "        net.eval()  # 设置为评估模式\n",
    "        if not device:\n",
    "            device = next(iter(net.parameters())).device\n",
    "    # 正确预测的数量，总预测的数量\n",
    "    metric = d2l.Accumulator(2)\n",
    "    with torch.no_grad():\n",
    "        for X, y in data_iter:\n",
    "            if isinstance(X, list):\n",
    "                # BERT微调所需的（之后将介绍）\n",
    "                X = [x.to(device) for x in X]\n",
    "            else:\n",
    "                X = X.to(device)\n",
    "            y = y.to(device)\n",
    "            metric.add(d2l.accuracy(net(X), y), y.numel())\n",
    "    return metric[0] / metric[1]\n",
    "\n",
    "# 定义用于训练模型的函数@save\n",
    "def train_ch6(net, train_iter, test_iter, num_epochs, lr, device):\n",
    "    \"\"\"用GPU训练模型(在第六章定义)\"\"\"\n",
    "    # 定义初始化权重的函数，使用Xavier初始化\n",
    "    def init_weights(m):\n",
    "        if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
    "            nn.init.xavier_uniform_(m.weight)\n",
    "    # 对模型的所有层应用初始化权重的函数\n",
    "    net.apply(init_weights)\n",
    "    # 打印在哪个设备上进行训练（CPU或GPU）\n",
    "    print('training on', device)\n",
    "    # 将模型移动到指定的设备（CPU或GPU）\n",
    "    net.to(device)\n",
    "    # 定义优化器，使用随机梯度下降（SGD）优化器\n",
    "    optimizer = torch.optim.SGD(net.parameters(), lr=lr)\n",
    "    # 定义损失函数，使用交叉熵损失\n",
    "    loss = nn.CrossEntropyLoss()\n",
    "    # 创建用于绘制训练过程的动画\n",
    "    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n",
    "                            legend=['train loss', 'train acc', 'test acc'])\n",
    "    # 创建计时器和获取每个epoch的批量数\n",
    "    timer, num_batches = d2l.Timer(), len(train_iter)\n",
    "    for epoch in range(num_epochs):\n",
    "        # 训练损失之和，训练准确率之和，样本数\n",
    "        metric = d2l.Accumulator(3)\n",
    "        # 设置模型为训练模式\n",
    "        net.train()\n",
    "        for i, (X, y) in enumerate(train_iter):\n",
    "            timer.start()\n",
    "            optimizer.zero_grad()\n",
    "            # 将输入数据和标签移动到指定的设备上\n",
    "            X, y = X.to(device), y.to(device)\n",
    "            # 前向传播\n",
    "            y_hat = net(X)\n",
    "            # 计算损失\n",
    "            l = loss(y_hat, y)\n",
    "            # 反向传播\n",
    "            l.backward()\n",
    "            # 更新模型参数\n",
    "            optimizer.step()\n",
    "            with torch.no_grad():\n",
    "                # 记录训练损失、训练准确率和样本数\n",
    "                metric.add(l * X.shape[0], d2l.accuracy(y_hat, y), X.shape[0])\n",
    "            timer.stop()\n",
    "            train_l = metric[0] / metric[2]\n",
    "            train_acc = metric[1] / metric[2]\n",
    "            # 每完成一定批量数的训练，更新动画\n",
    "            if (i + 1) % (num_batches // 5) == 0 or i == num_batches - 1:\n",
    "                animator.add(epoch + (i + 1) / num_batches,\n",
    "                            (train_l, train_acc, None))\n",
    "        # 计算并记录测试集上的准确率\n",
    "        test_acc = evaluate_accuracy_gpu(net, test_iter)\n",
    "        animator.add(epoch + 1, (None, None, test_acc))\n",
    "    # 打印最终的训练损失、训练准确率和测试准确率\n",
    "    print(f'loss {train_l:.3f}, train acc {train_acc:.3f}, '\n",
    "        f'test acc {test_acc:.3f}')\n",
    "    # 打印训练速度（样本/秒）\n",
    "    print(f'{metric[2] * num_epochs / timer.sum():.1f} examples/sec '\n",
    "        f'on {str(device)}')\n",
    "\n",
    "# 设置学习率和训练周期\n",
    "lr, num_epochs = 0.9, 10\n",
    "\n",
    "# 调用train_ch6函数来在GPU上训练模型，使用你之前定义的函数\n",
    "train_ch6(net, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hPIOeTz_umAj"
   },
   "source": [
    "**第3问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "I7wIit-_umHa"
   },
   "source": [
    "&emsp;&emsp; 将卷积层`nn.Sigmoid()`修改为`nn.ReLU()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.293, train acc 0.890, test acc 0.874\n",
      "134235.1 examples/sec on cuda:0\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-03-01T02:09:29.093950</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 238.965625 183.35625 \n",
       "L 238.965625 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "L 30.103125 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 51.803125 145.8 \n",
       "L 51.803125 7.2 \n",
       "\" clip-path=\"url(#p5461a4ba60)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"ma9d8665e81\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma9d8665e81\" x=\"51.803125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(48.621875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 95.203125 145.8 \n",
       "L 95.203125 7.2 \n",
       "\" clip-path=\"url(#p5461a4ba60)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma9d8665e81\" x=\"95.203125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 4 -->\n",
       "      <g transform=\"translate(92.021875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 138.603125 145.8 \n",
       "L 138.603125 7.2 \n",
       "\" clip-path=\"url(#p5461a4ba60)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma9d8665e81\" x=\"138.603125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 6 -->\n",
       "      <g transform=\"translate(135.421875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
       "Q 1688 2584 1439 2293 \n",
       "Q 1191 2003 1191 1497 \n",
       "Q 1191 994 1439 701 \n",
       "Q 1688 409 2113 409 \n",
       "Q 2538 409 2786 701 \n",
       "Q 3034 994 3034 1497 \n",
       "Q 3034 2003 2786 2293 \n",
       "Q 2538 2584 2113 2584 \n",
       "z\n",
       "M 3366 4563 \n",
       "L 3366 3988 \n",
       "Q 3128 4100 2886 4159 \n",
       "Q 2644 4219 2406 4219 \n",
       "Q 1781 4219 1451 3797 \n",
       "Q 1122 3375 1075 2522 \n",
       "Q 1259 2794 1537 2939 \n",
       "Q 1816 3084 2150 3084 \n",
       "Q 2853 3084 3261 2657 \n",
       "Q 3669 2231 3669 1497 \n",
       "Q 3669 778 3244 343 \n",
       "Q 2819 -91 2113 -91 \n",
       "Q 1303 -91 875 529 \n",
       "Q 447 1150 447 2328 \n",
       "Q 447 3434 972 4092 \n",
       "Q 1497 4750 2381 4750 \n",
       "Q 2619 4750 2861 4703 \n",
       "Q 3103 4656 3366 4563 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 182.003125 145.8 \n",
       "L 182.003125 7.2 \n",
       "\" clip-path=\"url(#p5461a4ba60)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma9d8665e81\" x=\"182.003125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 8 -->\n",
       "      <g transform=\"translate(178.821875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
       "Q 1584 2216 1326 1975 \n",
       "Q 1069 1734 1069 1313 \n",
       "Q 1069 891 1326 650 \n",
       "Q 1584 409 2034 409 \n",
       "Q 2484 409 2743 651 \n",
       "Q 3003 894 3003 1313 \n",
       "Q 3003 1734 2745 1975 \n",
       "Q 2488 2216 2034 2216 \n",
       "z\n",
       "M 1403 2484 \n",
       "Q 997 2584 770 2862 \n",
       "Q 544 3141 544 3541 \n",
       "Q 544 4100 942 4425 \n",
       "Q 1341 4750 2034 4750 \n",
       "Q 2731 4750 3128 4425 \n",
       "Q 3525 4100 3525 3541 \n",
       "Q 3525 3141 3298 2862 \n",
       "Q 3072 2584 2669 2484 \n",
       "Q 3125 2378 3379 2068 \n",
       "Q 3634 1759 3634 1313 \n",
       "Q 3634 634 3220 271 \n",
       "Q 2806 -91 2034 -91 \n",
       "Q 1263 -91 848 271 \n",
       "Q 434 634 434 1313 \n",
       "Q 434 1759 690 2068 \n",
       "Q 947 2378 1403 2484 \n",
       "z\n",
       "M 1172 3481 \n",
       "Q 1172 3119 1398 2916 \n",
       "Q 1625 2713 2034 2713 \n",
       "Q 2441 2713 2670 2916 \n",
       "Q 2900 3119 2900 3481 \n",
       "Q 2900 3844 2670 4047 \n",
       "Q 2441 4250 2034 4250 \n",
       "Q 1625 4250 1398 4047 \n",
       "Q 1172 3844 1172 3481 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" clip-path=\"url(#p5461a4ba60)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#ma9d8665e81\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 10 -->\n",
       "      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 30.103125 120.758959 \n",
       "L 225.403125 120.758959 \n",
       "\" clip-path=\"url(#p5461a4ba60)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <defs>\n",
       "       <path id=\"m865ef71622\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m865ef71622\" x=\"30.103125\" y=\"120.758959\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(7.2 124.558177) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 30.103125 88.201608 \n",
       "L 225.403125 88.201608 \n",
       "\" clip-path=\"url(#p5461a4ba60)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m865ef71622\" x=\"30.103125\" y=\"88.201608\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(7.2 92.000827) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 30.103125 55.644257 \n",
       "L 225.403125 55.644257 \n",
       "\" clip-path=\"url(#p5461a4ba60)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m865ef71622\" x=\"30.103125\" y=\"55.644257\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(7.2 59.443476) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 30.103125 23.086907 \n",
       "L 225.403125 23.086907 \n",
       "\" clip-path=\"url(#p5461a4ba60)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m865ef71622\" x=\"30.103125\" y=\"23.086907\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 2.0 -->\n",
       "      <g transform=\"translate(7.2 26.886126) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_19\">\n",
       "    <path d=\"M 12.743125 13.5 \n",
       "L 17.083125 44.938888 \n",
       "L 21.423125 60.039484 \n",
       "L 25.763125 69.956941 \n",
       "L 30.103125 76.640433 \n",
       "L 34.443125 106.904039 \n",
       "L 38.783125 109.748789 \n",
       "L 43.123125 111.177901 \n",
       "L 47.463125 112.58753 \n",
       "L 51.803125 113.784824 \n",
       "L 56.143125 120.512763 \n",
       "L 60.483125 121.609264 \n",
       "L 64.823125 121.80824 \n",
       "L 69.163125 122.246481 \n",
       "L 73.503125 122.367531 \n",
       "L 77.843125 125.548385 \n",
       "L 82.183125 125.548932 \n",
       "L 86.523125 126.016167 \n",
       "L 90.863125 126.101491 \n",
       "L 95.203125 126.323225 \n",
       "L 99.543125 127.559411 \n",
       "L 103.883125 128.363996 \n",
       "L 108.223125 128.409178 \n",
       "L 112.563125 128.505568 \n",
       "L 116.903125 128.596943 \n",
       "L 121.243125 130.229614 \n",
       "L 125.583125 130.415675 \n",
       "L 129.923125 130.266652 \n",
       "L 134.263125 130.287343 \n",
       "L 138.603125 130.323713 \n",
       "L 142.943125 132.082686 \n",
       "L 147.283125 131.314498 \n",
       "L 151.623125 131.76235 \n",
       "L 155.963125 131.840621 \n",
       "L 160.303125 131.74333 \n",
       "L 164.643125 133.221121 \n",
       "L 168.983125 132.629739 \n",
       "L 173.323125 132.689474 \n",
       "L 177.663125 132.619526 \n",
       "L 182.003125 132.660397 \n",
       "L 186.343125 132.805997 \n",
       "L 190.683125 133.242312 \n",
       "L 195.023125 133.362211 \n",
       "L 199.363125 133.603055 \n",
       "L 203.703125 133.577749 \n",
       "L 208.043125 133.918485 \n",
       "L 212.383125 134.066627 \n",
       "L 216.723125 134.165418 \n",
       "L 221.063125 134.020194 \n",
       "L 225.403125 134.234095 \n",
       "\" clip-path=\"url(#p5461a4ba60)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_20\">\n",
       "    <path d=\"M 12.743125 139.5 \n",
       "L 17.083125 128.649354 \n",
       "L 21.423125 123.311521 \n",
       "L 25.763125 119.776718 \n",
       "L 30.103125 117.391443 \n",
       "L 34.443125 106.212057 \n",
       "L 38.783125 104.89158 \n",
       "L 43.123125 104.238557 \n",
       "L 47.463125 103.725338 \n",
       "L 51.803125 103.210547 \n",
       "L 56.143125 100.464732 \n",
       "L 60.483125 100.072377 \n",
       "L 64.823125 100.066064 \n",
       "L 69.163125 99.896494 \n",
       "L 73.503125 99.834349 \n",
       "L 77.843125 98.732958 \n",
       "L 82.183125 98.651782 \n",
       "L 86.523125 98.401035 \n",
       "L 90.863125 98.362251 \n",
       "L 95.203125 98.277023 \n",
       "L 99.543125 97.980719 \n",
       "L 103.883125 97.569423 \n",
       "L 108.223125 97.607305 \n",
       "L 112.563125 97.576188 \n",
       "L 116.903125 97.504328 \n",
       "L 121.243125 96.779301 \n",
       "L 125.583125 96.76036 \n",
       "L 129.923125 96.813576 \n",
       "L 134.263125 96.794183 \n",
       "L 138.603125 96.773958 \n",
       "L 142.943125 96.119062 \n",
       "L 147.283125 96.538476 \n",
       "L 151.623125 96.330122 \n",
       "L 155.963125 96.27871 \n",
       "L 160.303125 96.299706 \n",
       "L 164.643125 95.669883 \n",
       "L 168.983125 95.872826 \n",
       "L 173.323125 95.871924 \n",
       "L 177.663125 95.880943 \n",
       "L 182.003125 95.878631 \n",
       "L 186.343125 95.816002 \n",
       "L 190.683125 95.594118 \n",
       "L 195.023125 95.541804 \n",
       "L 199.363125 95.445294 \n",
       "L 203.703125 95.47275 \n",
       "L 208.043125 95.437176 \n",
       "L 212.383125 95.429058 \n",
       "L 216.723125 95.383058 \n",
       "L 221.063125 95.476412 \n",
       "L 225.403125 95.387015 \n",
       "\" clip-path=\"url(#p5461a4ba60)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_21\">\n",
       "    <path d=\"M 30.103125 107.423468 \n",
       "L 51.803125 103.275661 \n",
       "L 73.503125 98.730655 \n",
       "L 95.203125 98.496242 \n",
       "L 116.903125 97.851607 \n",
       "L 138.603125 96.842329 \n",
       "L 160.303125 97.669286 \n",
       "L 182.003125 97.115811 \n",
       "L 203.703125 97.29162 \n",
       "L 225.403125 96.386526 \n",
       "\" clip-path=\"url(#p5461a4ba60)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 30.103125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 30.103125 7.2 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 140.634375 59.234375 \n",
       "L 218.403125 59.234375 \n",
       "Q 220.403125 59.234375 220.403125 57.234375 \n",
       "L 220.403125 14.2 \n",
       "Q 220.403125 12.2 218.403125 12.2 \n",
       "L 140.634375 12.2 \n",
       "Q 138.634375 12.2 138.634375 14.2 \n",
       "L 138.634375 57.234375 \n",
       "Q 138.634375 59.234375 140.634375 59.234375 \n",
       "z\n",
       "\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_22\">\n",
       "     <path d=\"M 142.634375 20.298438 \n",
       "L 152.634375 20.298438 \n",
       "L 162.634375 20.298438 \n",
       "\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_11\">\n",
       "     <!-- train loss -->\n",
       "     <g transform=\"translate(170.634375 23.798438) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_23\">\n",
       "     <path d=\"M 142.634375 34.976563 \n",
       "L 152.634375 34.976563 \n",
       "L 162.634375 34.976563 \n",
       "\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- train acc -->\n",
       "     <g transform=\"translate(170.634375 38.476563) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_24\">\n",
       "     <path d=\"M 142.634375 49.654688 \n",
       "L 152.634375 49.654688 \n",
       "L 162.634375 49.654688 \n",
       "\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- test acc -->\n",
       "     <g transform=\"translate(170.634375 53.154688) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p5461a4ba60\">\n",
       "   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义神经网络结构\n",
    "# 创建一个序列化的神经网络模型\n",
    "net = nn.Sequential(\n",
    "    # 第一个二维卷积层。输入通道为1（例如，用于灰度图像），输出通道为10。使用5x5的卷积核，边缘填充2个像素。\n",
    "    nn.Conv2d(1, 10, kernel_size=5, padding=2), \n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # 最大池化层，用于降低特征维度。使用2x2的窗口大小和步长2，减少数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第二个二维卷积层。输入通道为10（上一卷积层的输出通道数），输出通道为16。使用5x5的卷积核。\n",
    "    nn.Conv2d(10, 16, kernel_size=5), \n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # 又一个最大池化层，进一步减小数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # Flatten层，将多维度的数据展平成一维，以便传递给全连接层。\n",
    "    nn.Flatten(),\n",
    "    # 全连接（线性）层。输入特征数量为16*5*5（从上一层的输出尺寸计算而来），输出特征数量为120。\n",
    "    nn.Linear(16 * 5 * 5, 120), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第二个全连接（线性）层，进一步转换特征。输入特征数量为120，输出为84。\n",
    "    nn.Linear(120, 84), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第三个全连接（线性）层，最终输出特征数量为10，通常用于10类分类问题。\n",
    "    nn.Linear(84, 10))\n",
    "\n",
    "# 打印每个层的形状\n",
    "X = torch.rand(size=(1, 1, 28, 28), dtype=torch.float32)\n",
    "for layer in net:\n",
    "    X = layer(X)\n",
    "    print(layer.__class__.__name__,'output shape: \\t',X.shape)\n",
    "\n",
    "# 加载Fashion MNIST数据集\n",
    "batch_size = 256\n",
    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size=batch_size)\n",
    "\n",
    "# 定义用于在GPU上评估模型精度的函数\n",
    "def evaluate_accuracy_gpu(net, data_iter, device=None): #@save\n",
    "    \"\"\"使用GPU计算模型在数据集上的精度\"\"\"\n",
    "    if isinstance(net, nn.Module):\n",
    "        net.eval()  # 设置为评估模式\n",
    "        if not device:\n",
    "            device = next(iter(net.parameters())).device\n",
    "    # 正确预测的数量，总预测的数量\n",
    "    metric = d2l.Accumulator(2)\n",
    "    with torch.no_grad():\n",
    "        for X, y in data_iter:\n",
    "            if isinstance(X, list):\n",
    "                # BERT微调所需的（之后将介绍）\n",
    "                X = [x.to(device) for x in X]\n",
    "            else:\n",
    "                X = X.to(device)\n",
    "            y = y.to(device)\n",
    "            metric.add(d2l.accuracy(net(X), y), y.numel())\n",
    "    return metric[0] / metric[1]\n",
    "\n",
    "# 定义用于训练模型的函数@save\n",
    "def train_ch6(net, train_iter, test_iter, num_epochs, lr, device):\n",
    "    \"\"\"用GPU训练模型(在第六章定义)\"\"\"\n",
    "    # 定义初始化权重的函数，使用Xavier初始化\n",
    "    def init_weights(m):\n",
    "        if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
    "            nn.init.xavier_uniform_(m.weight)\n",
    "    # 对模型的所有层应用初始化权重的函数\n",
    "    net.apply(init_weights)\n",
    "    # 打印在哪个设备上进行训练（CPU或GPU）\n",
    "    print('training on', device)\n",
    "    # 将模型移动到指定的设备（CPU或GPU）\n",
    "    net.to(device)\n",
    "    # 定义优化器，使用随机梯度下降（SGD）优化器\n",
    "    optimizer = torch.optim.SGD(net.parameters(), lr=lr)\n",
    "    # 定义损失函数，使用交叉熵损失\n",
    "    loss = nn.CrossEntropyLoss()\n",
    "    # 创建用于绘制训练过程的动画\n",
    "    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n",
    "                            legend=['train loss', 'train acc', 'test acc'])\n",
    "    # 创建计时器和获取每个epoch的批量数\n",
    "    timer, num_batches = d2l.Timer(), len(train_iter)\n",
    "    for epoch in range(num_epochs):\n",
    "        # 训练损失之和，训练准确率之和，样本数\n",
    "        metric = d2l.Accumulator(3)\n",
    "        # 设置模型为训练模式\n",
    "        net.train()\n",
    "        for i, (X, y) in enumerate(train_iter):\n",
    "            timer.start()\n",
    "            optimizer.zero_grad()\n",
    "            # 将输入数据和标签移动到指定的设备上\n",
    "            X, y = X.to(device), y.to(device)\n",
    "            # 前向传播\n",
    "            y_hat = net(X)\n",
    "            # 计算损失\n",
    "            l = loss(y_hat, y)\n",
    "            # 反向传播\n",
    "            l.backward()\n",
    "            # 更新模型参数\n",
    "            optimizer.step()\n",
    "            with torch.no_grad():\n",
    "                # 记录训练损失、训练准确率和样本数\n",
    "                metric.add(l * X.shape[0], d2l.accuracy(y_hat, y), X.shape[0])\n",
    "            timer.stop()\n",
    "            train_l = metric[0] / metric[2]\n",
    "            train_acc = metric[1] / metric[2]\n",
    "            # 每完成一定批量数的训练，更新动画\n",
    "            if (i + 1) % (num_batches // 5) == 0 or i == num_batches - 1:\n",
    "                animator.add(epoch + (i + 1) / num_batches,\n",
    "                            (train_l, train_acc, None))\n",
    "        # 计算并记录测试集上的准确率\n",
    "        test_acc = evaluate_accuracy_gpu(net, test_iter)\n",
    "        animator.add(epoch + 1, (None, None, test_acc))\n",
    "    # 打印最终的训练损失、训练准确率和测试准确率\n",
    "    print(f'loss {train_l:.3f}, train acc {train_acc:.3f}, '\n",
    "        f'test acc {test_acc:.3f}')\n",
    "    # 打印训练速度（样本/秒）\n",
    "    print(f'{metric[2] * num_epochs / timer.sum():.1f} examples/sec '\n",
    "        f'on {str(device)}')\n",
    "\n",
    "# 设置学习率和训练周期\n",
    "lr, num_epochs = 0.9, 10\n",
    "\n",
    "# 调用train_ch6函数来在GPU上训练模型，使用你之前定义的函数\n",
    "train_ch6(net, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ev80H6ydumOY"
   },
   "source": [
    "**第4问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9YahX5BwumU5"
   },
   "source": [
    "&emsp;&emsp; 添加`conv3`为`nn.Cov2d(16, 120, kernel_size = 5), ReLU()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.274, train acc 0.895, test acc 0.865\n",
      "118062.6 examples/sec on cuda:0\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-03-01T02:09:57.268450</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 238.965625 183.35625 \n",
       "L 238.965625 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "L 30.103125 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 51.803125 145.8 \n",
       "L 51.803125 7.2 \n",
       "\" clip-path=\"url(#p20bc7874c4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"mefe8541811\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#mefe8541811\" x=\"51.803125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(48.621875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 95.203125 145.8 \n",
       "L 95.203125 7.2 \n",
       "\" clip-path=\"url(#p20bc7874c4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mefe8541811\" x=\"95.203125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 4 -->\n",
       "      <g transform=\"translate(92.021875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 138.603125 145.8 \n",
       "L 138.603125 7.2 \n",
       "\" clip-path=\"url(#p20bc7874c4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mefe8541811\" x=\"138.603125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 6 -->\n",
       "      <g transform=\"translate(135.421875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
       "Q 1688 2584 1439 2293 \n",
       "Q 1191 2003 1191 1497 \n",
       "Q 1191 994 1439 701 \n",
       "Q 1688 409 2113 409 \n",
       "Q 2538 409 2786 701 \n",
       "Q 3034 994 3034 1497 \n",
       "Q 3034 2003 2786 2293 \n",
       "Q 2538 2584 2113 2584 \n",
       "z\n",
       "M 3366 4563 \n",
       "L 3366 3988 \n",
       "Q 3128 4100 2886 4159 \n",
       "Q 2644 4219 2406 4219 \n",
       "Q 1781 4219 1451 3797 \n",
       "Q 1122 3375 1075 2522 \n",
       "Q 1259 2794 1537 2939 \n",
       "Q 1816 3084 2150 3084 \n",
       "Q 2853 3084 3261 2657 \n",
       "Q 3669 2231 3669 1497 \n",
       "Q 3669 778 3244 343 \n",
       "Q 2819 -91 2113 -91 \n",
       "Q 1303 -91 875 529 \n",
       "Q 447 1150 447 2328 \n",
       "Q 447 3434 972 4092 \n",
       "Q 1497 4750 2381 4750 \n",
       "Q 2619 4750 2861 4703 \n",
       "Q 3103 4656 3366 4563 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 182.003125 145.8 \n",
       "L 182.003125 7.2 \n",
       "\" clip-path=\"url(#p20bc7874c4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mefe8541811\" x=\"182.003125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 8 -->\n",
       "      <g transform=\"translate(178.821875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
       "Q 1584 2216 1326 1975 \n",
       "Q 1069 1734 1069 1313 \n",
       "Q 1069 891 1326 650 \n",
       "Q 1584 409 2034 409 \n",
       "Q 2484 409 2743 651 \n",
       "Q 3003 894 3003 1313 \n",
       "Q 3003 1734 2745 1975 \n",
       "Q 2488 2216 2034 2216 \n",
       "z\n",
       "M 1403 2484 \n",
       "Q 997 2584 770 2862 \n",
       "Q 544 3141 544 3541 \n",
       "Q 544 4100 942 4425 \n",
       "Q 1341 4750 2034 4750 \n",
       "Q 2731 4750 3128 4425 \n",
       "Q 3525 4100 3525 3541 \n",
       "Q 3525 3141 3298 2862 \n",
       "Q 3072 2584 2669 2484 \n",
       "Q 3125 2378 3379 2068 \n",
       "Q 3634 1759 3634 1313 \n",
       "Q 3634 634 3220 271 \n",
       "Q 2806 -91 2034 -91 \n",
       "Q 1263 -91 848 271 \n",
       "Q 434 634 434 1313 \n",
       "Q 434 1759 690 2068 \n",
       "Q 947 2378 1403 2484 \n",
       "z\n",
       "M 1172 3481 \n",
       "Q 1172 3119 1398 2916 \n",
       "Q 1625 2713 2034 2713 \n",
       "Q 2441 2713 2670 2916 \n",
       "Q 2900 3119 2900 3481 \n",
       "Q 2900 3844 2670 4047 \n",
       "Q 2441 4250 2034 4250 \n",
       "Q 1625 4250 1398 4047 \n",
       "Q 1172 3844 1172 3481 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" clip-path=\"url(#p20bc7874c4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#mefe8541811\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 10 -->\n",
       "      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 30.103125 118.627337 \n",
       "L 225.403125 118.627337 \n",
       "\" clip-path=\"url(#p20bc7874c4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <defs>\n",
       "       <path id=\"m1448eadc33\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m1448eadc33\" x=\"30.103125\" y=\"118.627337\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(7.2 122.426556) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 30.103125 89.499539 \n",
       "L 225.403125 89.499539 \n",
       "\" clip-path=\"url(#p20bc7874c4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m1448eadc33\" x=\"30.103125\" y=\"89.499539\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(7.2 93.298758) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 30.103125 60.37174 \n",
       "L 225.403125 60.37174 \n",
       "\" clip-path=\"url(#p20bc7874c4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m1448eadc33\" x=\"30.103125\" y=\"60.37174\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(7.2 64.170959) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 30.103125 31.243942 \n",
       "L 225.403125 31.243942 \n",
       "\" clip-path=\"url(#p20bc7874c4)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m1448eadc33\" x=\"30.103125\" y=\"31.243942\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 2.0 -->\n",
       "      <g transform=\"translate(7.2 35.043161) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_19\">\n",
       "    <path d=\"M 12.743125 13.5 \n",
       "L 17.083125 35.641823 \n",
       "L 21.423125 50.947929 \n",
       "L 25.763125 60.02877 \n",
       "L 30.103125 67.786913 \n",
       "L 34.443125 105.92116 \n",
       "L 38.783125 108.355274 \n",
       "L 43.123125 109.524036 \n",
       "L 47.463125 110.70377 \n",
       "L 51.803125 111.460341 \n",
       "L 56.143125 116.558064 \n",
       "L 60.483125 117.347446 \n",
       "L 64.823125 118.22586 \n",
       "L 69.163125 119.113135 \n",
       "L 73.503125 119.561583 \n",
       "L 77.843125 121.935638 \n",
       "L 82.183125 122.392484 \n",
       "L 86.523125 122.505455 \n",
       "L 90.863125 123.110987 \n",
       "L 95.203125 123.585166 \n",
       "L 99.543125 125.668938 \n",
       "L 103.883125 125.227152 \n",
       "L 108.223125 125.542115 \n",
       "L 112.563125 125.850431 \n",
       "L 116.903125 126.056222 \n",
       "L 121.243125 126.991556 \n",
       "L 125.583125 127.442116 \n",
       "L 129.923125 127.668507 \n",
       "L 134.263125 127.560755 \n",
       "L 138.603125 127.768697 \n",
       "L 142.943125 128.402296 \n",
       "L 147.283125 128.568996 \n",
       "L 151.623125 128.521037 \n",
       "L 155.963125 128.816916 \n",
       "L 160.303125 128.981888 \n",
       "L 164.643125 130.875174 \n",
       "L 168.983125 130.080136 \n",
       "L 173.323125 130.149433 \n",
       "L 177.663125 130.066544 \n",
       "L 182.003125 130.089054 \n",
       "L 186.343125 131.093411 \n",
       "L 190.683125 131.013052 \n",
       "L 195.023125 130.889651 \n",
       "L 199.363125 130.868732 \n",
       "L 203.703125 130.910221 \n",
       "L 208.043125 131.704997 \n",
       "L 212.383125 131.689692 \n",
       "L 216.723125 131.480312 \n",
       "L 221.063125 131.6637 \n",
       "L 225.403125 131.770337 \n",
       "\" clip-path=\"url(#p20bc7874c4)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_20\">\n",
       "    <path d=\"M 12.743125 139.5 \n",
       "L 17.083125 132.084903 \n",
       "L 21.423125 126.675893 \n",
       "L 25.763125 123.243919 \n",
       "L 30.103125 120.330343 \n",
       "L 34.443125 106.247055 \n",
       "L 38.783125 105.060833 \n",
       "L 43.123125 104.563749 \n",
       "L 47.463125 104.113066 \n",
       "L 51.803125 103.810997 \n",
       "L 56.143125 101.584476 \n",
       "L 60.483125 101.131775 \n",
       "L 64.823125 100.762998 \n",
       "L 69.163125 100.401886 \n",
       "L 73.503125 100.154487 \n",
       "L 77.843125 98.998997 \n",
       "L 82.183125 98.8949 \n",
       "L 86.523125 98.856973 \n",
       "L 90.863125 98.667339 \n",
       "L 95.203125 98.507796 \n",
       "L 99.543125 97.812775 \n",
       "L 103.883125 97.958027 \n",
       "L 108.223125 97.804706 \n",
       "L 112.563125 97.690522 \n",
       "L 116.903125 97.601921 \n",
       "L 121.243125 97.222085 \n",
       "L 125.583125 97.079254 \n",
       "L 129.923125 97.00098 \n",
       "L 134.263125 97.070781 \n",
       "L 138.603125 97.008685 \n",
       "L 142.943125 96.868639 \n",
       "L 147.283125 96.718546 \n",
       "L 151.623125 96.723388 \n",
       "L 155.963125 96.579347 \n",
       "L 160.303125 96.54167 \n",
       "L 164.643125 95.963237 \n",
       "L 168.983125 96.258582 \n",
       "L 173.323125 96.226304 \n",
       "L 177.663125 96.198061 \n",
       "L 182.003125 96.183398 \n",
       "L 186.343125 95.905137 \n",
       "L 190.683125 95.936608 \n",
       "L 195.023125 95.924504 \n",
       "L 199.363125 95.889401 \n",
       "L 203.703125 95.874643 \n",
       "L 208.043125 95.658209 \n",
       "L 212.383125 95.663051 \n",
       "L 216.723125 95.675962 \n",
       "L 221.063125 95.636421 \n",
       "L 225.403125 95.596958 \n",
       "\" clip-path=\"url(#p20bc7874c4)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_21\">\n",
       "    <path d=\"M 30.103125 108.432608 \n",
       "L 51.803125 105.834408 \n",
       "L 73.503125 102.10605 \n",
       "L 95.203125 100.422463 \n",
       "L 116.903125 99.397165 \n",
       "L 138.603125 97.585416 \n",
       "L 160.303125 96.734884 \n",
       "L 182.003125 102.438107 \n",
       "L 203.703125 97.276661 \n",
       "L 225.403125 97.393172 \n",
       "\" clip-path=\"url(#p20bc7874c4)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 30.103125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 30.103125 7.2 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 140.634375 59.234375 \n",
       "L 218.403125 59.234375 \n",
       "Q 220.403125 59.234375 220.403125 57.234375 \n",
       "L 220.403125 14.2 \n",
       "Q 220.403125 12.2 218.403125 12.2 \n",
       "L 140.634375 12.2 \n",
       "Q 138.634375 12.2 138.634375 14.2 \n",
       "L 138.634375 57.234375 \n",
       "Q 138.634375 59.234375 140.634375 59.234375 \n",
       "z\n",
       "\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_22\">\n",
       "     <path d=\"M 142.634375 20.298438 \n",
       "L 152.634375 20.298438 \n",
       "L 162.634375 20.298438 \n",
       "\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_11\">\n",
       "     <!-- train loss -->\n",
       "     <g transform=\"translate(170.634375 23.798438) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_23\">\n",
       "     <path d=\"M 142.634375 34.976563 \n",
       "L 152.634375 34.976563 \n",
       "L 162.634375 34.976563 \n",
       "\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- train acc -->\n",
       "     <g transform=\"translate(170.634375 38.476563) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_24\">\n",
       "     <path d=\"M 142.634375 49.654688 \n",
       "L 152.634375 49.654688 \n",
       "L 162.634375 49.654688 \n",
       "\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- test acc -->\n",
       "     <g transform=\"translate(170.634375 53.154688) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p20bc7874c4\">\n",
       "   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义神经网络结构\n",
    "# 创建一个序列化的神经网络模型\n",
    "net = nn.Sequential(\n",
    "    # 第一个二维卷积层。输入通道为1（例如，用于灰度图像），输出通道为10。使用5x5的卷积核，边缘填充2个像素。\n",
    "    nn.Conv2d(1, 10, kernel_size=5, padding=2), \n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # 最大池化层，用于降低特征维度。使用2x2的窗口大小和步长2，减少数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第二个二维卷积层。输入通道为10（上一卷积层的输出通道数），输出通道为16。使用5x5的卷积核。\n",
    "    nn.Conv2d(10, 16, kernel_size=5), \n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # 又一个最大池化层，进一步减小数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第三个二维卷积层。输入通道为16，输出通道为120。使用5x5的卷积核。\n",
    "    nn.Conv2d(16, 120, kernel_size=5),\n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # Flatten层，将多维度的数据展平成一维，以便传递给全连接层。\n",
    "    nn.Flatten(),\n",
    "    # 第一个全连接（线性）层。输入特征数量为120，输出特征数量为120。\n",
    "    nn.Linear(120, 120), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第二个全连接（线性）层，进一步转换特征。输入特征数量为120，输出为84。\n",
    "    nn.Linear(120, 84), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第三个全连接（线性）层，最终输出特征数量为10，通常用于10类分类问题。\n",
    "    nn.Linear(84, 10))\n",
    "\n",
    "# 打印每个层的形状\n",
    "X = torch.rand(size=(1, 1, 28, 28), dtype=torch.float32)\n",
    "for layer in net:\n",
    "    X = layer(X)\n",
    "    print(layer.__class__.__name__,'output shape: \\t',X.shape)\n",
    "\n",
    "# 加载Fashion MNIST数据集\n",
    "batch_size = 256\n",
    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size=batch_size)\n",
    "\n",
    "# 定义用于在GPU上评估模型精度的函数\n",
    "def evaluate_accuracy_gpu(net, data_iter, device=None): #@save\n",
    "    \"\"\"使用GPU计算模型在数据集上的精度\"\"\"\n",
    "    if isinstance(net, nn.Module):\n",
    "        net.eval()  # 设置为评估模式\n",
    "        if not device:\n",
    "            device = next(iter(net.parameters())).device\n",
    "    # 正确预测的数量，总预测的数量\n",
    "    metric = d2l.Accumulator(2)\n",
    "    with torch.no_grad():\n",
    "        for X, y in data_iter:\n",
    "            if isinstance(X, list):\n",
    "                # BERT微调所需的（之后将介绍）\n",
    "                X = [x.to(device) for x in X]\n",
    "            else:\n",
    "                X = X.to(device)\n",
    "            y = y.to(device)\n",
    "            metric.add(d2l.accuracy(net(X), y), y.numel())\n",
    "    return metric[0] / metric[1]\n",
    "\n",
    "# 定义用于训练模型的函数@save\n",
    "def train_ch6(net, train_iter, test_iter, num_epochs, lr, device):\n",
    "    \"\"\"用GPU训练模型(在第六章定义)\"\"\"\n",
    "    # 定义初始化权重的函数，使用Xavier初始化\n",
    "    def init_weights(m):\n",
    "        if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
    "            nn.init.xavier_uniform_(m.weight)\n",
    "    # 对模型的所有层应用初始化权重的函数\n",
    "    net.apply(init_weights)\n",
    "    # 打印在哪个设备上进行训练（CPU或GPU）\n",
    "    print('training on', device)\n",
    "    # 将模型移动到指定的设备（CPU或GPU）\n",
    "    net.to(device)\n",
    "    # 定义优化器，使用随机梯度下降（SGD）优化器\n",
    "    optimizer = torch.optim.SGD(net.parameters(), lr=lr)\n",
    "    # 定义损失函数，使用交叉熵损失\n",
    "    loss = nn.CrossEntropyLoss()\n",
    "    # 创建用于绘制训练过程的动画\n",
    "    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n",
    "                            legend=['train loss', 'train acc', 'test acc'])\n",
    "    # 创建计时器和获取每个epoch的批量数\n",
    "    timer, num_batches = d2l.Timer(), len(train_iter)\n",
    "    for epoch in range(num_epochs):\n",
    "        # 训练损失之和，训练准确率之和，样本数\n",
    "        metric = d2l.Accumulator(3)\n",
    "        # 设置模型为训练模式\n",
    "        net.train()\n",
    "        for i, (X, y) in enumerate(train_iter):\n",
    "            timer.start()\n",
    "            optimizer.zero_grad()\n",
    "            # 将输入数据和标签移动到指定的设备上\n",
    "            X, y = X.to(device), y.to(device)\n",
    "            # 前向传播\n",
    "            y_hat = net(X)\n",
    "            # 计算损失\n",
    "            l = loss(y_hat, y)\n",
    "            # 反向传播\n",
    "            l.backward()\n",
    "            # 更新模型参数\n",
    "            optimizer.step()\n",
    "            with torch.no_grad():\n",
    "                # 记录训练损失、训练准确率和样本数\n",
    "                metric.add(l * X.shape[0], d2l.accuracy(y_hat, y), X.shape[0])\n",
    "            timer.stop()\n",
    "            train_l = metric[0] / metric[2]\n",
    "            train_acc = metric[1] / metric[2]\n",
    "            # 每完成一定批量数的训练，更新动画\n",
    "            if (i + 1) % (num_batches // 5) == 0 or i == num_batches - 1:\n",
    "                animator.add(epoch + (i + 1) / num_batches,\n",
    "                            (train_l, train_acc, None))\n",
    "        # 计算并记录测试集上的准确率\n",
    "        test_acc = evaluate_accuracy_gpu(net, test_iter)\n",
    "        animator.add(epoch + 1, (None, None, test_acc))\n",
    "    # 打印最终的训练损失、训练准确率和测试准确率\n",
    "    print(f'loss {train_l:.3f}, train acc {train_acc:.3f}, '\n",
    "        f'test acc {test_acc:.3f}')\n",
    "    # 打印训练速度（样本/秒）\n",
    "    print(f'{metric[2] * num_epochs / timer.sum():.1f} examples/sec '\n",
    "        f'on {str(device)}')\n",
    "\n",
    "# 设置学习率和训练周期\n",
    "lr, num_epochs = 0.9, 10\n",
    "\n",
    "# 调用train_ch6函数来在GPU上训练模型，使用你之前定义的函数\n",
    "train_ch6(net, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BKPp_2wBumbv"
   },
   "source": [
    "**第5问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EDpiFmlBumh2"
   },
   "source": [
    "&emsp;&emsp; 添加`nn.Linear(84, 20),nn.Sigmoid()`并将`nn.Linear(84, 10)`替换为`nn.Linear(20, 10)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.300, train acc 0.891, test acc 0.867\n",
      "114410.0 examples/sec on cuda:0\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-03-01T02:10:26.293796</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 238.965625 183.35625 \n",
       "L 238.965625 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "L 30.103125 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 51.803125 145.8 \n",
       "L 51.803125 7.2 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"m9f1c4c8020\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m9f1c4c8020\" x=\"51.803125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(48.621875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 95.203125 145.8 \n",
       "L 95.203125 7.2 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m9f1c4c8020\" x=\"95.203125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 4 -->\n",
       "      <g transform=\"translate(92.021875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 138.603125 145.8 \n",
       "L 138.603125 7.2 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m9f1c4c8020\" x=\"138.603125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 6 -->\n",
       "      <g transform=\"translate(135.421875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-36\" d=\"M 2113 2584 \n",
       "Q 1688 2584 1439 2293 \n",
       "Q 1191 2003 1191 1497 \n",
       "Q 1191 994 1439 701 \n",
       "Q 1688 409 2113 409 \n",
       "Q 2538 409 2786 701 \n",
       "Q 3034 994 3034 1497 \n",
       "Q 3034 2003 2786 2293 \n",
       "Q 2538 2584 2113 2584 \n",
       "z\n",
       "M 3366 4563 \n",
       "L 3366 3988 \n",
       "Q 3128 4100 2886 4159 \n",
       "Q 2644 4219 2406 4219 \n",
       "Q 1781 4219 1451 3797 \n",
       "Q 1122 3375 1075 2522 \n",
       "Q 1259 2794 1537 2939 \n",
       "Q 1816 3084 2150 3084 \n",
       "Q 2853 3084 3261 2657 \n",
       "Q 3669 2231 3669 1497 \n",
       "Q 3669 778 3244 343 \n",
       "Q 2819 -91 2113 -91 \n",
       "Q 1303 -91 875 529 \n",
       "Q 447 1150 447 2328 \n",
       "Q 447 3434 972 4092 \n",
       "Q 1497 4750 2381 4750 \n",
       "Q 2619 4750 2861 4703 \n",
       "Q 3103 4656 3366 4563 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 182.003125 145.8 \n",
       "L 182.003125 7.2 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m9f1c4c8020\" x=\"182.003125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 8 -->\n",
       "      <g transform=\"translate(178.821875 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-38\" d=\"M 2034 2216 \n",
       "Q 1584 2216 1326 1975 \n",
       "Q 1069 1734 1069 1313 \n",
       "Q 1069 891 1326 650 \n",
       "Q 1584 409 2034 409 \n",
       "Q 2484 409 2743 651 \n",
       "Q 3003 894 3003 1313 \n",
       "Q 3003 1734 2745 1975 \n",
       "Q 2488 2216 2034 2216 \n",
       "z\n",
       "M 1403 2484 \n",
       "Q 997 2584 770 2862 \n",
       "Q 544 3141 544 3541 \n",
       "Q 544 4100 942 4425 \n",
       "Q 1341 4750 2034 4750 \n",
       "Q 2731 4750 3128 4425 \n",
       "Q 3525 4100 3525 3541 \n",
       "Q 3525 3141 3298 2862 \n",
       "Q 3072 2584 2669 2484 \n",
       "Q 3125 2378 3379 2068 \n",
       "Q 3634 1759 3634 1313 \n",
       "Q 3634 634 3220 271 \n",
       "Q 2806 -91 2034 -91 \n",
       "Q 1263 -91 848 271 \n",
       "Q 434 634 434 1313 \n",
       "Q 434 1759 690 2068 \n",
       "Q 947 2378 1403 2484 \n",
       "z\n",
       "M 1172 3481 \n",
       "Q 1172 3119 1398 2916 \n",
       "Q 1625 2713 2034 2713 \n",
       "Q 2441 2713 2670 2916 \n",
       "Q 2900 3119 2900 3481 \n",
       "Q 2900 3844 2670 4047 \n",
       "Q 2441 4250 2034 4250 \n",
       "Q 1625 4250 1398 4047 \n",
       "Q 1172 3844 1172 3481 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m9f1c4c8020\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 10 -->\n",
       "      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 30.103125 145.243994 \n",
       "L 225.403125 145.243994 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <defs>\n",
       "       <path id=\"maf17186b53\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#maf17186b53\" x=\"30.103125\" y=\"145.243994\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.0 -->\n",
       "      <g transform=\"translate(7.2 149.043213) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 30.103125 116.73255 \n",
       "L 225.403125 116.73255 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#maf17186b53\" x=\"30.103125\" y=\"116.73255\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(7.2 120.531769) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 30.103125 88.221106 \n",
       "L 225.403125 88.221106 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#maf17186b53\" x=\"30.103125\" y=\"88.221106\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(7.2 92.020325) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 30.103125 59.709662 \n",
       "L 225.403125 59.709662 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#maf17186b53\" x=\"30.103125\" y=\"59.709662\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(7.2 63.508881) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <path d=\"M 30.103125 31.198219 \n",
       "L 225.403125 31.198219 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#maf17186b53\" x=\"30.103125\" y=\"31.198219\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 2.0 -->\n",
       "      <g transform=\"translate(7.2 34.997437) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_21\">\n",
       "    <path d=\"M 12.743125 13.5 \n",
       "L 17.083125 15.874138 \n",
       "L 21.423125 24.096922 \n",
       "L 25.763125 31.557483 \n",
       "L 30.103125 39.994099 \n",
       "L 34.443125 85.528256 \n",
       "L 38.783125 86.706367 \n",
       "L 43.123125 88.867934 \n",
       "L 47.463125 88.247438 \n",
       "L 51.803125 89.808731 \n",
       "L 56.143125 97.868494 \n",
       "L 60.483125 100.232859 \n",
       "L 64.823125 102.11988 \n",
       "L 69.163125 103.519031 \n",
       "L 73.503125 104.70379 \n",
       "L 77.843125 111.871222 \n",
       "L 82.183125 112.51958 \n",
       "L 86.523125 112.775081 \n",
       "L 90.863125 113.347868 \n",
       "L 95.203125 114.087242 \n",
       "L 99.543125 118.43806 \n",
       "L 103.883125 118.445093 \n",
       "L 108.223125 119.022661 \n",
       "L 112.563125 119.271326 \n",
       "L 116.903125 119.444259 \n",
       "L 121.243125 121.629626 \n",
       "L 125.583125 122.048879 \n",
       "L 129.923125 121.848549 \n",
       "L 134.263125 122.323707 \n",
       "L 138.603125 122.500602 \n",
       "L 142.943125 123.858458 \n",
       "L 147.283125 124.107998 \n",
       "L 151.623125 124.074559 \n",
       "L 155.963125 124.241067 \n",
       "L 160.303125 124.356313 \n",
       "L 164.643125 126.1036 \n",
       "L 168.983125 125.831655 \n",
       "L 173.323125 125.682661 \n",
       "L 177.663125 125.584393 \n",
       "L 182.003125 125.855667 \n",
       "L 186.343125 126.771865 \n",
       "L 190.683125 126.846606 \n",
       "L 195.023125 126.77349 \n",
       "L 199.363125 126.901256 \n",
       "L 203.703125 127.016557 \n",
       "L 208.043125 128.002776 \n",
       "L 212.383125 128.084156 \n",
       "L 216.723125 127.994408 \n",
       "L 221.063125 128.057728 \n",
       "L 225.403125 128.138387 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_22\">\n",
       "    <path d=\"M 12.743125 139.5 \n",
       "L 17.083125 137.824668 \n",
       "L 21.423125 135.207802 \n",
       "L 25.763125 132.555786 \n",
       "L 30.103125 129.355517 \n",
       "L 34.443125 111.41983 \n",
       "L 38.783125 111.246846 \n",
       "L 43.123125 110.467236 \n",
       "L 47.463125 110.732636 \n",
       "L 51.803125 110.197727 \n",
       "L 56.143125 106.950699 \n",
       "L 60.483125 106.038389 \n",
       "L 64.823125 105.38358 \n",
       "L 69.163125 104.819212 \n",
       "L 73.503125 104.235985 \n",
       "L 77.843125 100.936566 \n",
       "L 82.183125 100.58349 \n",
       "L 86.523125 100.396289 \n",
       "L 90.863125 100.058616 \n",
       "L 95.203125 99.736879 \n",
       "L 99.543125 97.60012 \n",
       "L 103.883125 97.607229 \n",
       "L 108.223125 97.481639 \n",
       "L 112.563125 97.431876 \n",
       "L 116.903125 97.415097 \n",
       "L 121.243125 96.590656 \n",
       "L 125.583125 96.467435 \n",
       "L 129.923125 96.521147 \n",
       "L 134.263125 96.388052 \n",
       "L 138.603125 96.358273 \n",
       "L 142.943125 95.893983 \n",
       "L 147.283125 95.806307 \n",
       "L 151.623125 95.833953 \n",
       "L 155.963125 95.796828 \n",
       "L 160.303125 95.756681 \n",
       "L 164.643125 95.211529 \n",
       "L 168.983125 95.249443 \n",
       "L 173.323125 95.391621 \n",
       "L 177.663125 95.456786 \n",
       "L 182.003125 95.35657 \n",
       "L 186.343125 94.927172 \n",
       "L 190.683125 94.882149 \n",
       "L 195.023125 94.936651 \n",
       "L 199.363125 94.931912 \n",
       "L 203.703125 94.862372 \n",
       "L 208.043125 94.571727 \n",
       "L 212.383125 94.48642 \n",
       "L 216.723125 94.495899 \n",
       "L 221.063125 94.487605 \n",
       "L 225.403125 94.456559 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_23\">\n",
       "    <path d=\"M 30.103125 112.119399 \n",
       "L 51.803125 108.099285 \n",
       "L 73.503125 104.273049 \n",
       "L 95.203125 100.047653 \n",
       "L 116.903125 97.949211 \n",
       "L 138.603125 100.384088 \n",
       "L 160.303125 96.57496 \n",
       "L 182.003125 95.531441 \n",
       "L 203.703125 96.090265 \n",
       "L 225.403125 95.805151 \n",
       "\" clip-path=\"url(#p0760c22076)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 30.103125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 30.103125 7.2 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 140.634375 59.234375 \n",
       "L 218.403125 59.234375 \n",
       "Q 220.403125 59.234375 220.403125 57.234375 \n",
       "L 220.403125 14.2 \n",
       "Q 220.403125 12.2 218.403125 12.2 \n",
       "L 140.634375 12.2 \n",
       "Q 138.634375 12.2 138.634375 14.2 \n",
       "L 138.634375 57.234375 \n",
       "Q 138.634375 59.234375 140.634375 59.234375 \n",
       "z\n",
       "\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_24\">\n",
       "     <path d=\"M 142.634375 20.298438 \n",
       "L 152.634375 20.298438 \n",
       "L 162.634375 20.298438 \n",
       "\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- train loss -->\n",
       "     <g transform=\"translate(170.634375 23.798438) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_25\">\n",
       "     <path d=\"M 142.634375 34.976563 \n",
       "L 152.634375 34.976563 \n",
       "L 162.634375 34.976563 \n",
       "\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- train acc -->\n",
       "     <g transform=\"translate(170.634375 38.476563) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_26\">\n",
       "     <path d=\"M 142.634375 49.654688 \n",
       "L 152.634375 49.654688 \n",
       "L 162.634375 49.654688 \n",
       "\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- test acc -->\n",
       "     <g transform=\"translate(170.634375 53.154688) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p0760c22076\">\n",
       "   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义神经网络结构\n",
    "# 创建一个序列化的神经网络模型\n",
    "net = nn.Sequential(\n",
    "    # 第一个二维卷积层。输入通道为1（例如，用于灰度图像），输出通道为10。使用5x5的卷积核，边缘填充2个像素。\n",
    "    nn.Conv2d(1, 10, kernel_size=5, padding=2), \n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # 最大池化层，用于降低特征维度。使用2x2的窗口大小和步长2，减少数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第二个二维卷积层。输入通道为10（上一卷积层的输出通道数），输出通道为16。使用5x5的卷积核。\n",
    "    nn.Conv2d(10, 16, kernel_size=5), \n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # 又一个最大池化层，进一步减小数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第三个二维卷积层。输入通道为16（上一卷积层的输出通道数），输出通道为120。使用5x5的卷积核。\n",
    "    nn.Conv2d(16, 120, kernel_size = 5),\n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # Flatten层，将多维度的数据展平成一维，以便传递给全连接层。\n",
    "    nn.Flatten(),\n",
    "    # 全连接（线性）层。输入特征数量为16*5*5（从上一层的输出尺寸计算而来），输出特征数量为120。\n",
    "    nn.Linear(120, 120), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第二个全连接（线性）层，进一步转换特征。输入特征数量为120，输出为84。\n",
    "    nn.Linear(120, 84), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第三个全连接（线性）层，进一步转换特征。输入特征数量为84，输出为20。\n",
    "    nn.Linear(84, 20),\n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第四个全连接（线性）层，最终输出特征数量为10，通常用于10类分类问题。\n",
    "    nn.Linear(20, 10))\n",
    "\n",
    "# 打印每个层的形状\n",
    "X = torch.rand(size=(1, 1, 28, 28), dtype=torch.float32)\n",
    "for layer in net:\n",
    "    X = layer(X)\n",
    "    print(layer.__class__.__name__,'output shape: \\t',X.shape)\n",
    "\n",
    "# 加载Fashion MNIST数据集\n",
    "batch_size = 256\n",
    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size=batch_size)\n",
    "\n",
    "# 定义用于在GPU上评估模型精度的函数\n",
    "def evaluate_accuracy_gpu(net, data_iter, device=None): #@save\n",
    "    \"\"\"使用GPU计算模型在数据集上的精度\"\"\"\n",
    "    if isinstance(net, nn.Module):\n",
    "        net.eval()  # 设置为评估模式\n",
    "        if not device:\n",
    "            device = next(iter(net.parameters())).device\n",
    "    # 正确预测的数量，总预测的数量\n",
    "    metric = d2l.Accumulator(2)\n",
    "    with torch.no_grad():\n",
    "        for X, y in data_iter:\n",
    "            if isinstance(X, list):\n",
    "                # BERT微调所需的（之后将介绍）\n",
    "                X = [x.to(device) for x in X]\n",
    "            else:\n",
    "                X = X.to(device)\n",
    "            y = y.to(device)\n",
    "            metric.add(d2l.accuracy(net(X), y), y.numel())\n",
    "    return metric[0] / metric[1]\n",
    "\n",
    "# 定义用于训练模型的函数@save\n",
    "def train_ch6(net, train_iter, test_iter, num_epochs, lr, device):\n",
    "    \"\"\"用GPU训练模型(在第六章定义)\"\"\"\n",
    "    # 定义初始化权重的函数，使用Xavier初始化\n",
    "    def init_weights(m):\n",
    "        if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
    "            nn.init.xavier_uniform_(m.weight)\n",
    "    # 对模型的所有层应用初始化权重的函数\n",
    "    net.apply(init_weights)\n",
    "    # 打印在哪个设备上进行训练（CPU或GPU）\n",
    "    print('training on', device)\n",
    "    # 将模型移动到指定的设备（CPU或GPU）\n",
    "    net.to(device)\n",
    "    # 定义优化器，使用随机梯度下降（SGD）优化器\n",
    "    optimizer = torch.optim.SGD(net.parameters(), lr=lr)\n",
    "    # 定义损失函数，使用交叉熵损失\n",
    "    loss = nn.CrossEntropyLoss()\n",
    "    # 创建用于绘制训练过程的动画\n",
    "    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n",
    "                            legend=['train loss', 'train acc', 'test acc'])\n",
    "    # 创建计时器和获取每个epoch的批量数\n",
    "    timer, num_batches = d2l.Timer(), len(train_iter)\n",
    "    for epoch in range(num_epochs):\n",
    "        # 训练损失之和，训练准确率之和，样本数\n",
    "        metric = d2l.Accumulator(3)\n",
    "        # 设置模型为训练模式\n",
    "        net.train()\n",
    "        for i, (X, y) in enumerate(train_iter):\n",
    "            timer.start()\n",
    "            optimizer.zero_grad()\n",
    "            # 将输入数据和标签移动到指定的设备上\n",
    "            X, y = X.to(device), y.to(device)\n",
    "            # 前向传播\n",
    "            y_hat = net(X)\n",
    "            # 计算损失\n",
    "            l = loss(y_hat, y)\n",
    "            # 反向传播\n",
    "            l.backward()\n",
    "            # 更新模型参数\n",
    "            optimizer.step()\n",
    "            with torch.no_grad():\n",
    "                # 记录训练损失、训练准确率和样本数\n",
    "                metric.add(l * X.shape[0], d2l.accuracy(y_hat, y), X.shape[0])\n",
    "            timer.stop()\n",
    "            train_l = metric[0] / metric[2]\n",
    "            train_acc = metric[1] / metric[2]\n",
    "            # 每完成一定批量数的训练，更新动画\n",
    "            if (i + 1) % (num_batches // 5) == 0 or i == num_batches - 1:\n",
    "                animator.add(epoch + (i + 1) / num_batches,\n",
    "                            (train_l, train_acc, None))\n",
    "        # 计算并记录测试集上的准确率\n",
    "        test_acc = evaluate_accuracy_gpu(net, test_iter)\n",
    "        animator.add(epoch + 1, (None, None, test_acc))\n",
    "    # 打印最终的训练损失、训练准确率和测试准确率\n",
    "    print(f'loss {train_l:.3f}, train acc {train_acc:.3f}, '\n",
    "        f'test acc {test_acc:.3f}')\n",
    "    # 打印训练速度（样本/秒）\n",
    "    print(f'{metric[2] * num_epochs / timer.sum():.1f} examples/sec '\n",
    "        f'on {str(device)}')\n",
    "\n",
    "# 设置学习率和训练周期\n",
    "lr, num_epochs = 0.9, 10\n",
    "\n",
    "# 调用train_ch6函数来在GPU上训练模型，使用你之前定义的函数\n",
    "train_ch6(net, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BuBEDHOfumpI"
   },
   "source": [
    "**第6问：**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Va4i3jXKumvh"
   },
   "source": [
    "&emsp;&emsp; 修改为`lr, num_epochs = 0.1, 50`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.228, train acc 0.916, test acc 0.890\n",
      "111873.2 examples/sec on cuda:0\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-03-01T02:12:49.588210</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 238.965625 183.35625 \n",
       "L 238.965625 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "L 30.103125 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 65.974554 145.8 \n",
       "L 65.974554 7.2 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"m393e405532\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m393e405532\" x=\"65.974554\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 10 -->\n",
       "      <g transform=\"translate(59.612054 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 105.831696 145.8 \n",
       "L 105.831696 7.2 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m393e405532\" x=\"105.831696\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 20 -->\n",
       "      <g transform=\"translate(99.469196 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 145.688839 145.8 \n",
       "L 145.688839 7.2 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m393e405532\" x=\"145.688839\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 30 -->\n",
       "      <g transform=\"translate(139.326339 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
       "Q 3050 2419 3304 2112 \n",
       "Q 3559 1806 3559 1356 \n",
       "Q 3559 666 3084 287 \n",
       "Q 2609 -91 1734 -91 \n",
       "Q 1441 -91 1130 -33 \n",
       "Q 819 25 488 141 \n",
       "L 488 750 \n",
       "Q 750 597 1062 519 \n",
       "Q 1375 441 1716 441 \n",
       "Q 2309 441 2620 675 \n",
       "Q 2931 909 2931 1356 \n",
       "Q 2931 1769 2642 2001 \n",
       "Q 2353 2234 1838 2234 \n",
       "L 1294 2234 \n",
       "L 1294 2753 \n",
       "L 1863 2753 \n",
       "Q 2328 2753 2575 2939 \n",
       "Q 2822 3125 2822 3475 \n",
       "Q 2822 3834 2567 4026 \n",
       "Q 2313 4219 1838 4219 \n",
       "Q 1578 4219 1281 4162 \n",
       "Q 984 4106 628 3988 \n",
       "L 628 4550 \n",
       "Q 988 4650 1302 4700 \n",
       "Q 1616 4750 1894 4750 \n",
       "Q 2613 4750 3031 4423 \n",
       "Q 3450 4097 3450 3541 \n",
       "Q 3450 3153 3228 2886 \n",
       "Q 3006 2619 2597 2516 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-33\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 185.545982 145.8 \n",
       "L 185.545982 7.2 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m393e405532\" x=\"185.545982\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 40 -->\n",
       "      <g transform=\"translate(179.183482 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m393e405532\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 50 -->\n",
       "      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-35\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 30.103125 145.064869 \n",
       "L 225.403125 145.064869 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <defs>\n",
       "       <path id=\"m65cdfa7f6f\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m65cdfa7f6f\" x=\"30.103125\" y=\"145.064869\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.0 -->\n",
       "      <g transform=\"translate(7.2 148.864088) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 30.103125 116.765416 \n",
       "L 225.403125 116.765416 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m65cdfa7f6f\" x=\"30.103125\" y=\"116.765416\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(7.2 120.564634) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 30.103125 88.465962 \n",
       "L 225.403125 88.465962 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m65cdfa7f6f\" x=\"30.103125\" y=\"88.465962\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(7.2 92.265181) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 30.103125 60.166508 \n",
       "L 225.403125 60.166508 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m65cdfa7f6f\" x=\"30.103125\" y=\"60.166508\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(7.2 63.965727) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <path d=\"M 30.103125 31.867055 \n",
       "L 225.403125 31.867055 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m65cdfa7f6f\" x=\"30.103125\" y=\"31.867055\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 2.0 -->\n",
       "      <g transform=\"translate(7.2 35.666273) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_21\">\n",
       "    <path d=\"M 26.914554 13.5 \n",
       "L 27.711696 14.083417 \n",
       "L 29.305982 14.406799 \n",
       "L 30.103125 14.474572 \n",
       "L 30.900268 14.742672 \n",
       "L 35.683125 14.852339 \n",
       "L 38.074554 15.027974 \n",
       "L 38.871696 15.961898 \n",
       "L 39.668839 16.715046 \n",
       "L 40.465982 18.475442 \n",
       "L 41.263125 21.603684 \n",
       "L 42.060268 25.36455 \n",
       "L 42.857411 46.996778 \n",
       "L 43.654554 49.265504 \n",
       "L 46.045982 54.786563 \n",
       "L 46.843125 64.885851 \n",
       "L 47.640268 66.654088 \n",
       "L 48.437411 68.028943 \n",
       "L 50.031696 69.914025 \n",
       "L 50.828839 76.309581 \n",
       "L 53.220268 78.04172 \n",
       "L 54.017411 78.782914 \n",
       "L 54.814554 83.021275 \n",
       "L 55.611696 83.705941 \n",
       "L 56.408839 84.188294 \n",
       "L 57.205982 84.471719 \n",
       "L 58.003125 84.959817 \n",
       "L 58.800268 89.189243 \n",
       "L 61.191696 90.464933 \n",
       "L 61.988839 91.049704 \n",
       "L 62.785982 94.297825 \n",
       "L 63.583125 95.005206 \n",
       "L 64.380268 95.461245 \n",
       "L 65.177411 95.650713 \n",
       "L 65.974554 96.16257 \n",
       "L 66.771696 98.515096 \n",
       "L 67.568839 98.292578 \n",
       "L 68.365982 99.35678 \n",
       "L 69.163125 99.630647 \n",
       "L 69.960268 100.251158 \n",
       "L 70.757411 102.708918 \n",
       "L 71.554554 102.948585 \n",
       "L 72.351696 103.742436 \n",
       "L 73.148839 104.065233 \n",
       "L 73.945982 104.144893 \n",
       "L 74.743125 106.613293 \n",
       "L 75.540268 107.165199 \n",
       "L 76.337411 107.169338 \n",
       "L 77.931696 108.024844 \n",
       "L 78.728839 109.481807 \n",
       "L 79.525982 110.341047 \n",
       "L 81.917411 110.505002 \n",
       "L 82.714554 111.676728 \n",
       "L 83.511696 112.293913 \n",
       "L 85.105982 112.678045 \n",
       "L 85.903125 112.689963 \n",
       "L 87.497411 113.823408 \n",
       "L 89.888839 114.524831 \n",
       "L 90.685982 115.248447 \n",
       "L 91.483125 115.131825 \n",
       "L 93.077411 115.907154 \n",
       "L 93.874554 116.182652 \n",
       "L 94.671696 117.366443 \n",
       "L 95.468839 117.302799 \n",
       "L 97.860268 117.617927 \n",
       "L 99.454554 118.636564 \n",
       "L 101.845982 119.013347 \n",
       "L 102.643125 119.885382 \n",
       "L 103.440268 119.475282 \n",
       "L 104.237411 119.787047 \n",
       "L 105.831696 120.013612 \n",
       "L 107.425982 120.537176 \n",
       "L 108.223125 120.762502 \n",
       "L 109.020268 120.726737 \n",
       "L 109.817411 120.864944 \n",
       "L 110.614554 121.517688 \n",
       "L 111.411696 121.657408 \n",
       "L 113.005982 121.696792 \n",
       "L 113.803125 121.749073 \n",
       "L 114.600268 123.132496 \n",
       "L 115.397411 122.476828 \n",
       "L 116.991696 122.68921 \n",
       "L 118.585982 122.493322 \n",
       "L 119.383125 122.609464 \n",
       "L 120.180268 123.018218 \n",
       "L 120.977411 123.219242 \n",
       "L 121.774554 123.288378 \n",
       "L 122.571696 123.536476 \n",
       "L 123.368839 123.915036 \n",
       "L 124.165982 124.064026 \n",
       "L 126.557411 124.069558 \n",
       "L 128.151696 124.490842 \n",
       "L 128.948839 124.682555 \n",
       "L 129.745982 124.681924 \n",
       "L 130.543125 125.142137 \n",
       "L 132.137411 125.433979 \n",
       "L 133.731696 125.376958 \n",
       "L 135.325982 125.701185 \n",
       "L 136.123125 125.700638 \n",
       "L 137.717411 125.920927 \n",
       "L 138.514554 127.025549 \n",
       "L 139.311696 126.15947 \n",
       "L 140.108839 126.489364 \n",
       "L 140.905982 126.346792 \n",
       "L 141.703125 126.398438 \n",
       "L 142.500268 126.641613 \n",
       "L 143.297411 127.021738 \n",
       "L 144.094554 127.062028 \n",
       "L 144.891696 126.896615 \n",
       "L 145.688839 126.906047 \n",
       "L 146.485982 127.128999 \n",
       "L 149.674554 127.199187 \n",
       "L 150.471696 126.753905 \n",
       "L 151.268839 127.348677 \n",
       "L 152.065982 127.481605 \n",
       "L 152.863125 127.458747 \n",
       "L 153.660268 127.620671 \n",
       "L 154.457411 127.368691 \n",
       "L 155.254554 127.774837 \n",
       "L 157.645982 127.942804 \n",
       "L 158.443125 128.324762 \n",
       "L 161.631696 128.31673 \n",
       "L 162.428839 128.913955 \n",
       "L 163.225982 128.997408 \n",
       "L 164.023125 128.754794 \n",
       "L 164.820268 128.676628 \n",
       "L 165.617411 128.721441 \n",
       "L 166.414554 128.980039 \n",
       "L 168.008839 128.836053 \n",
       "L 170.400268 129.098915 \n",
       "L 171.197411 129.274083 \n",
       "L 173.588839 129.250109 \n",
       "L 174.385982 129.598686 \n",
       "L 178.371696 129.422657 \n",
       "L 179.168839 129.32787 \n",
       "L 180.763125 129.731475 \n",
       "L 181.560268 129.668664 \n",
       "L 182.357411 130.21679 \n",
       "L 183.951696 130.220655 \n",
       "L 184.748839 130.059845 \n",
       "L 185.545982 130.037494 \n",
       "L 186.343125 130.217978 \n",
       "L 187.140268 130.189372 \n",
       "L 187.937411 130.304382 \n",
       "L 190.328839 130.307996 \n",
       "L 191.125982 130.462809 \n",
       "L 192.720268 130.55697 \n",
       "L 193.517411 130.393257 \n",
       "L 194.314554 130.387765 \n",
       "L 195.908839 130.716737 \n",
       "L 196.705982 130.845289 \n",
       "L 199.097411 130.970107 \n",
       "L 199.894554 131.168969 \n",
       "L 200.691696 131.016377 \n",
       "L 201.488839 130.980955 \n",
       "L 202.285982 131.285259 \n",
       "L 204.677411 131.110735 \n",
       "L 205.474554 131.178604 \n",
       "L 206.271696 131.443813 \n",
       "L 207.068839 131.380583 \n",
       "L 207.865982 131.497859 \n",
       "L 209.460268 131.312149 \n",
       "L 210.257411 131.576583 \n",
       "L 211.054554 131.365792 \n",
       "L 211.851696 131.598236 \n",
       "L 214.243125 131.566811 \n",
       "L 218.228839 131.887511 \n",
       "L 219.025982 131.816381 \n",
       "L 219.823125 132.051009 \n",
       "L 221.417411 131.912492 \n",
       "L 222.214554 132.266049 \n",
       "L 223.011696 132.255208 \n",
       "L 223.808839 132.044713 \n",
       "L 225.403125 132.135666 \n",
       "L 225.403125 132.135666 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_22\">\n",
       "    <path d=\"M 26.914554 139.260094 \n",
       "L 27.711696 139.5 \n",
       "L 29.305982 139.302431 \n",
       "L 34.088839 139.087081 \n",
       "L 34.885982 139.462368 \n",
       "L 35.683125 138.728539 \n",
       "L 38.074554 137.76078 \n",
       "L 38.871696 132.387504 \n",
       "L 39.668839 130.964535 \n",
       "L 40.465982 130.534116 \n",
       "L 41.263125 129.866143 \n",
       "L 42.060268 128.840793 \n",
       "L 42.857411 122.029227 \n",
       "L 43.654554 121.921034 \n",
       "L 45.248839 120.768546 \n",
       "L 46.045982 120.589615 \n",
       "L 46.843125 118.524723 \n",
       "L 47.640268 118.400067 \n",
       "L 50.031696 117.537047 \n",
       "L 50.828839 115.184861 \n",
       "L 51.625982 115.050796 \n",
       "L 52.423125 114.731706 \n",
       "L 54.017411 114.44486 \n",
       "L 54.814554 112.536491 \n",
       "L 55.611696 112.327162 \n",
       "L 56.408839 111.823047 \n",
       "L 58.003125 111.369653 \n",
       "L 58.800268 109.173109 \n",
       "L 59.597411 109.363622 \n",
       "L 61.988839 108.586874 \n",
       "L 62.785982 107.489065 \n",
       "L 63.583125 107.136263 \n",
       "L 65.974554 106.697413 \n",
       "L 66.771696 105.663901 \n",
       "L 67.568839 105.781502 \n",
       "L 68.365982 105.339323 \n",
       "L 69.960268 104.968317 \n",
       "L 70.757411 104.191539 \n",
       "L 72.351696 103.627056 \n",
       "L 73.945982 103.498632 \n",
       "L 74.743125 102.860298 \n",
       "L 75.540268 102.669785 \n",
       "L 76.337411 102.651753 \n",
       "L 77.134554 102.367551 \n",
       "L 77.931696 102.253456 \n",
       "L 79.525982 101.343248 \n",
       "L 81.917411 101.406359 \n",
       "L 83.511696 100.73878 \n",
       "L 86.700268 100.555323 \n",
       "L 87.497411 100.256617 \n",
       "L 89.888839 100.005536 \n",
       "L 90.685982 99.86383 \n",
       "L 91.483125 99.887351 \n",
       "L 93.077411 99.480452 \n",
       "L 93.874554 99.338612 \n",
       "L 94.671696 98.885392 \n",
       "L 96.265982 98.669006 \n",
       "L 97.063125 98.480845 \n",
       "L 97.860268 98.42737 \n",
       "L 100.251696 97.836393 \n",
       "L 101.845982 97.717997 \n",
       "L 102.643125 97.257797 \n",
       "L 103.440268 97.46007 \n",
       "L 106.628839 97.215461 \n",
       "L 109.817411 96.965231 \n",
       "L 110.614554 96.726241 \n",
       "L 113.803125 96.630354 \n",
       "L 114.600268 96.091197 \n",
       "L 115.397411 96.443999 \n",
       "L 125.760268 95.870042 \n",
       "L 126.557411 96.15235 \n",
       "L 127.354554 95.877164 \n",
       "L 129.745982 95.678549 \n",
       "L 130.543125 95.700762 \n",
       "L 131.340268 95.498489 \n",
       "L 133.731696 95.427628 \n",
       "L 134.528839 95.296216 \n",
       "L 137.717411 95.248398 \n",
       "L 138.514554 94.976342 \n",
       "L 139.311696 95.237415 \n",
       "L 140.108839 95.035926 \n",
       "L 141.703125 95.054075 \n",
       "L 143.297411 94.915189 \n",
       "L 144.891696 94.902253 \n",
       "L 145.688839 94.918237 \n",
       "L 146.485982 94.774068 \n",
       "L 149.674554 94.778627 \n",
       "L 150.471696 94.966934 \n",
       "L 151.268839 94.727028 \n",
       "L 152.065982 94.643923 \n",
       "L 154.457411 94.78818 \n",
       "L 155.254554 94.623539 \n",
       "L 157.645982 94.561664 \n",
       "L 159.240268 94.346001 \n",
       "L 160.037411 94.455762 \n",
       "L 163.225982 94.230753 \n",
       "L 164.820268 94.348353 \n",
       "L 169.603125 94.147549 \n",
       "L 170.400268 93.974383 \n",
       "L 172.791696 94.090808 \n",
       "L 181.560268 94.004165 \n",
       "L 182.357411 93.762702 \n",
       "L 185.545982 93.828708 \n",
       "L 186.343125 93.687437 \n",
       "L 187.140268 93.760349 \n",
       "L 188.734554 93.722717 \n",
       "L 191.125982 93.755645 \n",
       "L 194.314554 93.701549 \n",
       "L 197.503125 93.520244 \n",
       "L 198.300268 93.649805 \n",
       "L 205.474554 93.433459 \n",
       "L 206.271696 93.339339 \n",
       "L 209.460268 93.421196 \n",
       "L 210.257411 93.348747 \n",
       "L 211.851696 93.337771 \n",
       "L 215.040268 93.245258 \n",
       "L 216.634554 93.231146 \n",
       "L 219.025982 93.254666 \n",
       "L 219.823125 93.130793 \n",
       "L 224.605982 93.226442 \n",
       "L 225.403125 93.197631 \n",
       "L 225.403125 93.197631 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_23\">\n",
       "    <path d=\"M 30.103125 139.404979 \n",
       "L 34.088839 139.404979 \n",
       "L 38.074554 132.188618 \n",
       "L 42.060268 122.097033 \n",
       "L 46.045982 118.486022 \n",
       "L 50.031696 115.605138 \n",
       "L 54.017411 113.709075 \n",
       "L 58.003125 109.458497 \n",
       "L 61.988839 108.994386 \n",
       "L 65.974554 107.545454 \n",
       "L 69.960268 104.823046 \n",
       "L 73.945982 103.855205 \n",
       "L 77.931696 102.372314 \n",
       "L 81.917411 101.930842 \n",
       "L 85.903125 101.093178 \n",
       "L 89.888839 100.48191 \n",
       "L 93.874554 100.204575 \n",
       "L 97.860268 99.106557 \n",
       "L 101.845982 98.455669 \n",
       "L 105.831696 97.997218 \n",
       "L 109.817411 97.108615 \n",
       "L 113.803125 97.719883 \n",
       "L 117.788839 96.85392 \n",
       "L 121.774554 96.955798 \n",
       "L 125.760268 96.650164 \n",
       "L 129.745982 97.629325 \n",
       "L 133.731696 95.931358 \n",
       "L 137.717411 95.727602 \n",
       "L 141.703125 96.135114 \n",
       "L 145.688839 96.633184 \n",
       "L 149.674554 95.455927 \n",
       "L 153.660268 96.027576 \n",
       "L 157.645982 96.163413 \n",
       "L 161.631696 95.574785 \n",
       "L 165.617411 95.144633 \n",
       "L 169.603125 95.28047 \n",
       "L 173.588839 95.580445 \n",
       "L 177.574554 95.246511 \n",
       "L 181.560268 94.980496 \n",
       "L 185.545982 95.478567 \n",
       "L 189.531696 95.693642 \n",
       "L 193.517411 94.816359 \n",
       "L 197.503125 95.382348 \n",
       "L 201.488839 94.980496 \n",
       "L 205.474554 94.952197 \n",
       "L 209.460268 94.844659 \n",
       "L 213.445982 95.580445 \n",
       "L 217.431696 94.80504 \n",
       "L 221.417411 95.014456 \n",
       "L 225.403125 94.663542 \n",
       "\" clip-path=\"url(#pe82b82794c)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 30.103125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 30.103125 7.2 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 140.634375 59.234375 \n",
       "L 218.403125 59.234375 \n",
       "Q 220.403125 59.234375 220.403125 57.234375 \n",
       "L 220.403125 14.2 \n",
       "Q 220.403125 12.2 218.403125 12.2 \n",
       "L 140.634375 12.2 \n",
       "Q 138.634375 12.2 138.634375 14.2 \n",
       "L 138.634375 57.234375 \n",
       "Q 138.634375 59.234375 140.634375 59.234375 \n",
       "z\n",
       "\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_24\">\n",
       "     <path d=\"M 142.634375 20.298438 \n",
       "L 152.634375 20.298438 \n",
       "L 162.634375 20.298438 \n",
       "\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- train loss -->\n",
       "     <g transform=\"translate(170.634375 23.798438) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_25\">\n",
       "     <path d=\"M 142.634375 34.976563 \n",
       "L 152.634375 34.976563 \n",
       "L 162.634375 34.976563 \n",
       "\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- train acc -->\n",
       "     <g transform=\"translate(170.634375 38.476563) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_26\">\n",
       "     <path d=\"M 142.634375 49.654688 \n",
       "L 152.634375 49.654688 \n",
       "L 162.634375 49.654688 \n",
       "\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- test acc -->\n",
       "     <g transform=\"translate(170.634375 53.154688) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pe82b82794c\">\n",
       "   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义神经网络结构\n",
    "# 创建一个序列化的神经网络模型\n",
    "net = nn.Sequential(\n",
    "    # 第一个二维卷积层。输入通道为1（例如，用于灰度图像），输出通道为10。使用5x5的卷积核，边缘填充2个像素。\n",
    "    nn.Conv2d(1, 10, kernel_size=5, padding=2), \n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # 最大池化层，用于降低特征维度。使用2x2的窗口大小和步长2，减少数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第二个二维卷积层。输入通道为10（上一卷积层的输出通道数），输出通道为16。使用5x5的卷积核。\n",
    "    nn.Conv2d(10, 16, kernel_size=5), \n",
    "    # 使用ReLU激活函数。。\n",
    "    nn.ReLU(),\n",
    "    # 又一个最大池化层，进一步减小数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第三个二维卷积层。输入通道为16（上一卷积层的输出通道数），输出通道为120。使用5x5的卷积核。\n",
    "    nn.Conv2d(16, 120, kernel_size = 5),\n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # Flatten层，将多维度的数据展平成一维，以便传递给全连接层。\n",
    "    nn.Flatten(),\n",
    "    # 全连接（线性）层。输入特征数量为16*5*5（从上一层的输出尺寸计算而来），输出特征数量为120。\n",
    "    nn.Linear(120, 120), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第二个全连接（线性）层，进一步转换特征。输入特征数量为120，输出为84。\n",
    "    nn.Linear(120, 84), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第三个全连接（线性）层，进一步转换特征。输入特征数量为84，输出为20。\n",
    "    nn.Linear(84, 20),\n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第四个全连接（线性）层，最终输出特征数量为10，通常用于10类分类问题。\n",
    "    nn.Linear(20, 10))\n",
    "\n",
    "X = torch.rand(size=(1, 1, 28, 28), dtype=torch.float32)\n",
    "for layer in net:\n",
    "    X = layer(X)\n",
    "    print(layer.__class__.__name__,'output shape: \\t',X.shape)\n",
    "\n",
    "# 加载Fashion MNIST数据集\n",
    "batch_size = 256\n",
    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size=batch_size)\n",
    "\n",
    "# 定义用于在GPU上评估模型精度的函数\n",
    "def evaluate_accuracy_gpu(net, data_iter, device=None): #@save\n",
    "    \"\"\"使用GPU计算模型在数据集上的精度\"\"\"\n",
    "    if isinstance(net, nn.Module):\n",
    "        net.eval()  # 设置为评估模式\n",
    "        if not device:\n",
    "            device = next(iter(net.parameters())).device\n",
    "    # 正确预测的数量，总预测的数量\n",
    "    metric = d2l.Accumulator(2)\n",
    "    with torch.no_grad():\n",
    "        for X, y in data_iter:\n",
    "            if isinstance(X, list):\n",
    "                # BERT微调所需的（之后将介绍）\n",
    "                X = [x.to(device) for x in X]\n",
    "            else:\n",
    "                X = X.to(device)\n",
    "            y = y.to(device)\n",
    "            metric.add(d2l.accuracy(net(X), y), y.numel())\n",
    "    return metric[0] / metric[1]\n",
    "\n",
    "# 定义用于训练模型的函数@save\n",
    "def train_ch6(net, train_iter, test_iter, num_epochs, lr, device):\n",
    "    \"\"\"用GPU训练模型(在第六章定义)\"\"\"\n",
    "    # 定义初始化权重的函数，使用Xavier初始化\n",
    "    def init_weights(m):\n",
    "        if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
    "            nn.init.xavier_uniform_(m.weight)\n",
    "    # 对模型的所有层应用初始化权重的函数\n",
    "    net.apply(init_weights)\n",
    "    # 打印在哪个设备上进行训练（CPU或GPU）\n",
    "    print('training on', device)\n",
    "    # 将模型移动到指定的设备（CPU或GPU）\n",
    "    net.to(device)\n",
    "    # 定义优化器，使用随机梯度下降（SGD）优化器\n",
    "    optimizer = torch.optim.SGD(net.parameters(), lr=lr)\n",
    "    # 定义损失函数，使用交叉熵损失\n",
    "    loss = nn.CrossEntropyLoss()\n",
    "    # 创建用于绘制训练过程的动画\n",
    "    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n",
    "                            legend=['train loss', 'train acc', 'test acc'])\n",
    "    # 创建计时器和获取每个epoch的批量数\n",
    "    timer, num_batches = d2l.Timer(), len(train_iter)\n",
    "    for epoch in range(num_epochs):\n",
    "        # 训练损失之和，训练准确率之和，样本数\n",
    "        metric = d2l.Accumulator(3)\n",
    "        # 设置模型为训练模式\n",
    "        net.train()\n",
    "        for i, (X, y) in enumerate(train_iter):\n",
    "            timer.start()\n",
    "            optimizer.zero_grad()\n",
    "            # 将输入数据和标签移动到指定的设备上\n",
    "            X, y = X.to(device), y.to(device)\n",
    "            # 前向传播\n",
    "            y_hat = net(X)\n",
    "            # 计算损失\n",
    "            l = loss(y_hat, y)\n",
    "            # 反向传播\n",
    "            l.backward()\n",
    "            # 更新模型参数\n",
    "            optimizer.step()\n",
    "            with torch.no_grad():\n",
    "                # 记录训练损失、训练准确率和样本数\n",
    "                metric.add(l * X.shape[0], d2l.accuracy(y_hat, y), X.shape[0])\n",
    "            timer.stop()\n",
    "            train_l = metric[0] / metric[2]\n",
    "            train_acc = metric[1] / metric[2]\n",
    "            # 每完成一定批量数的训练，更新动画\n",
    "            if (i + 1) % (num_batches // 5) == 0 or i == num_batches - 1:\n",
    "                animator.add(epoch + (i + 1) / num_batches,\n",
    "                            (train_l, train_acc, None))\n",
    "        # 计算并记录测试集上的准确率\n",
    "        test_acc = evaluate_accuracy_gpu(net, test_iter)\n",
    "        animator.add(epoch + 1, (None, None, test_acc))\n",
    "    # 打印最终的训练损失、训练准确率和测试准确率\n",
    "    print(f'loss {train_l:.3f}, train acc {train_acc:.3f}, '\n",
    "        f'test acc {test_acc:.3f}')\n",
    "    # 打印训练速度（样本/秒）\n",
    "    print(f'{metric[2] * num_epochs / timer.sum():.1f} examples/sec '\n",
    "        f'on {str(device)}')\n",
    "\n",
    "# 设置学习率和训练周期\n",
    "lr, num_epochs = 0.1, 50\n",
    "\n",
    "# 调用train_ch6函数来在GPU上训练模型，使用你之前定义的函数\n",
    "train_ch6(net, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "uv0dRt72H6Gu"
   },
   "source": [
    "### 练习 6.6.3\n",
    "\n",
    "在`MNIST`数据集上尝试以上改进的网络。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Rbz9BNXqH6Gu"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "&emsp;&emsp;使用练习 $6.6.2$ 改进后的神经网络结构：\\\n",
    "&emsp;&emsp;1.调整后卷积窗口的大小为 $5$ 。\\\n",
    "&emsp;&emsp;2.调整后输出通道的数量为 $10$ 。\\\n",
    "&emsp;&emsp;3.调整后卷积层激活函数使用`ReLU`函数。\\\n",
    "&emsp;&emsp;4.增加一个卷积层数量。\\\n",
    "&emsp;&emsp;5. 增加一个全连接层数量。\\\n",
    "&emsp;&emsp;6. 调整学习率和其他训练细节（学习率为 $0.1$ ，训练周期为 $50$）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "id": "082f2fe4",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.242, train acc 0.912, test acc 0.883\n",
      "111331.2 examples/sec on cuda:0\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-03-01T02:15:15.780465</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 238.965625 183.35625 \n",
       "L 238.965625 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "L 30.103125 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 65.974554 145.8 \n",
       "L 65.974554 7.2 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"m0e5d0bd945\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m0e5d0bd945\" x=\"65.974554\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 10 -->\n",
       "      <g transform=\"translate(59.612054 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 105.831696 145.8 \n",
       "L 105.831696 7.2 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m0e5d0bd945\" x=\"105.831696\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 20 -->\n",
       "      <g transform=\"translate(99.469196 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 145.688839 145.8 \n",
       "L 145.688839 7.2 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m0e5d0bd945\" x=\"145.688839\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 30 -->\n",
       "      <g transform=\"translate(139.326339 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
       "Q 3050 2419 3304 2112 \n",
       "Q 3559 1806 3559 1356 \n",
       "Q 3559 666 3084 287 \n",
       "Q 2609 -91 1734 -91 \n",
       "Q 1441 -91 1130 -33 \n",
       "Q 819 25 488 141 \n",
       "L 488 750 \n",
       "Q 750 597 1062 519 \n",
       "Q 1375 441 1716 441 \n",
       "Q 2309 441 2620 675 \n",
       "Q 2931 909 2931 1356 \n",
       "Q 2931 1769 2642 2001 \n",
       "Q 2353 2234 1838 2234 \n",
       "L 1294 2234 \n",
       "L 1294 2753 \n",
       "L 1863 2753 \n",
       "Q 2328 2753 2575 2939 \n",
       "Q 2822 3125 2822 3475 \n",
       "Q 2822 3834 2567 4026 \n",
       "Q 2313 4219 1838 4219 \n",
       "Q 1578 4219 1281 4162 \n",
       "Q 984 4106 628 3988 \n",
       "L 628 4550 \n",
       "Q 988 4650 1302 4700 \n",
       "Q 1616 4750 1894 4750 \n",
       "Q 2613 4750 3031 4423 \n",
       "Q 3450 4097 3450 3541 \n",
       "Q 3450 3153 3228 2886 \n",
       "Q 3006 2619 2597 2516 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-33\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 185.545982 145.8 \n",
       "L 185.545982 7.2 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m0e5d0bd945\" x=\"185.545982\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 40 -->\n",
       "      <g transform=\"translate(179.183482 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m0e5d0bd945\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 50 -->\n",
       "      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-35\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 30.103125 145.151704 \n",
       "L 225.403125 145.151704 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <defs>\n",
       "       <path id=\"m2e81c414a3\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2e81c414a3\" x=\"30.103125\" y=\"145.151704\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.0 -->\n",
       "      <g transform=\"translate(7.2 148.950923) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 30.103125 116.711174 \n",
       "L 225.403125 116.711174 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2e81c414a3\" x=\"30.103125\" y=\"116.711174\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(7.2 120.510393) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 30.103125 88.270644 \n",
       "L 225.403125 88.270644 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2e81c414a3\" x=\"30.103125\" y=\"88.270644\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(7.2 92.069862) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 30.103125 59.830113 \n",
       "L 225.403125 59.830113 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2e81c414a3\" x=\"30.103125\" y=\"59.830113\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(7.2 63.629332) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <path d=\"M 30.103125 31.389583 \n",
       "L 225.403125 31.389583 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m2e81c414a3\" x=\"30.103125\" y=\"31.389583\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 2.0 -->\n",
       "      <g transform=\"translate(7.2 35.188801) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_21\">\n",
       "    <path d=\"M 26.914554 13.5 \n",
       "L 27.711696 13.815472 \n",
       "L 29.305982 13.983929 \n",
       "L 38.074554 14.433951 \n",
       "L 38.871696 15.562597 \n",
       "L 39.668839 17.089701 \n",
       "L 40.465982 20.490355 \n",
       "L 42.060268 28.268751 \n",
       "L 42.857411 47.542809 \n",
       "L 44.451696 51.003286 \n",
       "L 46.045982 53.743674 \n",
       "L 46.843125 61.315704 \n",
       "L 48.437411 64.458883 \n",
       "L 50.031696 66.9231 \n",
       "L 50.828839 73.964808 \n",
       "L 51.625982 75.348181 \n",
       "L 52.423125 75.798441 \n",
       "L 54.017411 77.576779 \n",
       "L 54.814554 82.606928 \n",
       "L 55.611696 82.951681 \n",
       "L 58.003125 84.825451 \n",
       "L 58.800268 88.940736 \n",
       "L 60.394554 89.848295 \n",
       "L 61.191696 90.049766 \n",
       "L 61.988839 90.828937 \n",
       "L 62.785982 94.78931 \n",
       "L 63.583125 94.556526 \n",
       "L 65.974554 96.522246 \n",
       "L 66.771696 99.099743 \n",
       "L 67.568839 99.5907 \n",
       "L 69.163125 100.269985 \n",
       "L 69.960268 100.623103 \n",
       "L 70.757411 102.339341 \n",
       "L 72.351696 103.519398 \n",
       "L 73.945982 104.20607 \n",
       "L 74.743125 106.073507 \n",
       "L 75.540268 106.377567 \n",
       "L 76.337411 106.52237 \n",
       "L 77.931696 107.172902 \n",
       "L 78.728839 108.937402 \n",
       "L 79.525982 109.056136 \n",
       "L 81.120268 109.6385 \n",
       "L 81.917411 109.913535 \n",
       "L 82.714554 111.105499 \n",
       "L 83.511696 111.033778 \n",
       "L 85.105982 111.578859 \n",
       "L 85.903125 111.804 \n",
       "L 87.497411 113.309178 \n",
       "L 88.294554 113.360267 \n",
       "L 90.685982 113.994134 \n",
       "L 91.483125 114.543021 \n",
       "L 92.280268 114.672193 \n",
       "L 93.874554 115.261879 \n",
       "L 94.671696 116.426121 \n",
       "L 95.468839 116.267353 \n",
       "L 96.265982 116.30082 \n",
       "L 97.063125 116.583474 \n",
       "L 97.860268 116.535104 \n",
       "L 98.657411 117.263977 \n",
       "L 99.454554 117.616362 \n",
       "L 100.251696 117.58088 \n",
       "L 101.845982 118.020743 \n",
       "L 102.643125 118.615233 \n",
       "L 103.440268 118.671503 \n",
       "L 105.831696 119.154719 \n",
       "L 106.628839 119.896376 \n",
       "L 107.425982 120.424041 \n",
       "L 108.223125 120.491639 \n",
       "L 109.817411 120.266724 \n",
       "L 110.614554 121.406504 \n",
       "L 111.411696 121.367099 \n",
       "L 113.005982 121.118995 \n",
       "L 113.803125 121.095568 \n",
       "L 114.600268 122.082901 \n",
       "L 115.397411 122.030302 \n",
       "L 116.194554 122.130678 \n",
       "L 117.788839 122.016406 \n",
       "L 118.585982 121.717172 \n",
       "L 119.383125 122.497151 \n",
       "L 120.977411 122.736503 \n",
       "L 121.774554 122.886368 \n",
       "L 122.571696 123.765147 \n",
       "L 123.368839 123.542477 \n",
       "L 125.760268 123.591733 \n",
       "L 126.557411 124.011169 \n",
       "L 127.354554 123.886541 \n",
       "L 129.745982 124.051116 \n",
       "L 130.543125 124.767487 \n",
       "L 132.934554 124.785096 \n",
       "L 133.731696 124.906994 \n",
       "L 134.528839 124.831194 \n",
       "L 137.717411 125.358465 \n",
       "L 138.514554 125.878535 \n",
       "L 140.905982 125.708477 \n",
       "L 141.703125 125.942566 \n",
       "L 142.500268 125.469124 \n",
       "L 144.094554 126.17337 \n",
       "L 145.688839 126.271876 \n",
       "L 146.485982 126.623066 \n",
       "L 147.283125 126.229514 \n",
       "L 148.080268 126.547412 \n",
       "L 149.674554 126.679923 \n",
       "L 150.471696 127.186383 \n",
       "L 151.268839 127.031508 \n",
       "L 152.065982 127.266275 \n",
       "L 152.863125 127.085518 \n",
       "L 153.660268 127.073918 \n",
       "L 154.457411 127.970127 \n",
       "L 156.051696 127.55114 \n",
       "L 156.848839 127.594373 \n",
       "L 157.645982 127.498886 \n",
       "L 158.443125 127.057372 \n",
       "L 159.240268 127.494625 \n",
       "L 160.037411 127.497091 \n",
       "L 160.834554 127.806796 \n",
       "L 161.631696 127.733457 \n",
       "L 162.428839 127.962666 \n",
       "L 166.414554 127.999458 \n",
       "L 167.211696 128.194384 \n",
       "L 168.008839 128.577627 \n",
       "L 169.603125 128.349521 \n",
       "L 170.400268 128.885716 \n",
       "L 173.588839 128.701825 \n",
       "L 174.385982 129.11008 \n",
       "L 175.980268 128.79219 \n",
       "L 176.777411 129.001985 \n",
       "L 178.371696 129.062021 \n",
       "L 179.965982 129.259864 \n",
       "L 181.560268 129.157553 \n",
       "L 183.154554 129.335387 \n",
       "L 184.748839 129.463653 \n",
       "L 185.545982 129.520455 \n",
       "L 186.343125 129.864613 \n",
       "L 187.937411 129.778022 \n",
       "L 189.531696 129.608202 \n",
       "L 190.328839 130.070187 \n",
       "L 191.125982 129.924419 \n",
       "L 191.923125 129.632239 \n",
       "L 192.720268 129.776565 \n",
       "L 193.517411 129.792159 \n",
       "L 194.314554 129.389681 \n",
       "L 195.908839 130.150008 \n",
       "L 197.503125 130.094406 \n",
       "L 198.300268 130.845318 \n",
       "L 199.097411 130.547381 \n",
       "L 200.691696 130.433829 \n",
       "L 201.488839 130.341843 \n",
       "L 202.285982 130.576759 \n",
       "L 203.083125 130.437955 \n",
       "L 204.677411 130.512248 \n",
       "L 205.474554 130.553539 \n",
       "L 206.271696 131.119608 \n",
       "L 207.068839 131.266917 \n",
       "L 209.460268 130.763459 \n",
       "L 210.257411 130.848023 \n",
       "L 211.054554 131.317296 \n",
       "L 212.648839 131.214302 \n",
       "L 213.445982 131.012546 \n",
       "L 214.243125 131.292311 \n",
       "L 215.040268 130.996024 \n",
       "L 216.634554 130.997756 \n",
       "L 219.025982 131.013242 \n",
       "L 219.823125 131.28175 \n",
       "L 220.620268 131.374884 \n",
       "L 221.417411 131.347647 \n",
       "L 223.011696 131.593269 \n",
       "L 223.808839 131.449328 \n",
       "L 224.605982 131.479215 \n",
       "L 225.403125 131.391474 \n",
       "L 225.403125 131.391474 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_22\">\n",
       "    <path d=\"M 26.914554 139.407814 \n",
       "L 27.711696 139.5 \n",
       "L 29.305982 139.434997 \n",
       "L 33.291696 139.33572 \n",
       "L 34.088839 139.202894 \n",
       "L 34.885982 139.351084 \n",
       "L 35.683125 139.299082 \n",
       "L 37.277411 138.91852 \n",
       "L 38.074554 138.444479 \n",
       "L 38.871696 133.531554 \n",
       "L 39.668839 132.725519 \n",
       "L 41.263125 131.860389 \n",
       "L 42.060268 131.014865 \n",
       "L 42.857411 124.260962 \n",
       "L 43.654554 123.469109 \n",
       "L 44.451696 123.537658 \n",
       "L 45.248839 122.755259 \n",
       "L 46.045982 122.317751 \n",
       "L 46.843125 119.145827 \n",
       "L 48.437411 117.67558 \n",
       "L 49.234554 117.373021 \n",
       "L 50.031696 116.917842 \n",
       "L 50.828839 114.683084 \n",
       "L 51.625982 114.132333 \n",
       "L 52.423125 114.238701 \n",
       "L 54.017411 113.909782 \n",
       "L 54.814554 111.761501 \n",
       "L 55.611696 111.742591 \n",
       "L 56.408839 111.578705 \n",
       "L 58.003125 110.819244 \n",
       "L 58.800268 108.485356 \n",
       "L 59.597411 108.686274 \n",
       "L 60.394554 108.64609 \n",
       "L 61.191696 108.727639 \n",
       "L 61.988839 108.484277 \n",
       "L 62.785982 106.641638 \n",
       "L 63.583125 107.182934 \n",
       "L 65.177411 106.925287 \n",
       "L 65.974554 106.763624 \n",
       "L 66.771696 106.324897 \n",
       "L 67.568839 106.071976 \n",
       "L 68.365982 105.967184 \n",
       "L 69.163125 105.737507 \n",
       "L 69.960268 105.676248 \n",
       "L 70.757411 104.939744 \n",
       "L 72.351696 104.588335 \n",
       "L 73.945982 104.541471 \n",
       "L 74.743125 103.772056 \n",
       "L 77.134554 103.626686 \n",
       "L 77.931696 103.430394 \n",
       "L 78.728839 102.755648 \n",
       "L 81.120268 102.39045 \n",
       "L 81.917411 102.220724 \n",
       "L 82.714554 101.635235 \n",
       "L 83.511696 101.715602 \n",
       "L 85.903125 101.381728 \n",
       "L 87.497411 100.699193 \n",
       "L 89.091696 100.553823 \n",
       "L 89.888839 100.432762 \n",
       "L 90.685982 100.585734 \n",
       "L 91.483125 99.997162 \n",
       "L 93.874554 99.479057 \n",
       "L 94.671696 98.90275 \n",
       "L 96.265982 98.863354 \n",
       "L 97.063125 98.730197 \n",
       "L 97.860268 98.752875 \n",
       "L 98.657411 98.013984 \n",
       "L 99.454554 97.933616 \n",
       "L 100.251696 98.084896 \n",
       "L 105.831696 97.529932 \n",
       "L 107.425982 96.978665 \n",
       "L 109.020268 97.042486 \n",
       "L 109.817411 97.077728 \n",
       "L 110.614554 96.794293 \n",
       "L 111.411696 96.702107 \n",
       "L 113.803125 96.8521 \n",
       "L 114.600268 96.38773 \n",
       "L 115.397411 96.451551 \n",
       "L 116.194554 96.384578 \n",
       "L 117.788839 96.461516 \n",
       "L 118.585982 96.817931 \n",
       "L 119.383125 96.477552 \n",
       "L 120.977411 96.300271 \n",
       "L 121.774554 96.204604 \n",
       "L 122.571696 95.919709 \n",
       "L 124.165982 95.894496 \n",
       "L 125.760268 95.90977 \n",
       "L 127.354554 95.7117 \n",
       "L 129.745982 95.726803 \n",
       "L 145.688839 94.958908 \n",
       "L 149.674554 94.855574 \n",
       "L 150.471696 94.577104 \n",
       "L 151.268839 94.655108 \n",
       "L 152.065982 94.54086 \n",
       "L 152.863125 94.677563 \n",
       "L 153.660268 94.693463 \n",
       "L 154.457411 94.369095 \n",
       "L 156.051696 94.573953 \n",
       "L 157.645982 94.549365 \n",
       "L 158.443125 94.657471 \n",
       "L 160.037411 94.542436 \n",
       "L 160.834554 94.464827 \n",
       "L 161.631696 94.5086 \n",
       "L 162.428839 94.411642 \n",
       "L 164.820268 94.423461 \n",
       "L 166.414554 94.388005 \n",
       "L 168.008839 94.079143 \n",
       "L 169.603125 94.241259 \n",
       "L 170.400268 93.919984 \n",
       "L 171.994554 94.183147 \n",
       "L 175.183125 94.014534 \n",
       "L 175.980268 94.088598 \n",
       "L 179.168839 93.841981 \n",
       "L 183.154554 94.080719 \n",
       "L 187.140268 93.782887 \n",
       "L 189.531696 93.860156 \n",
       "L 190.328839 93.494511 \n",
       "L 191.923125 93.793918 \n",
       "L 193.517411 93.74355 \n",
       "L 194.314554 93.948349 \n",
       "L 197.503125 93.715109 \n",
       "L 198.300268 93.262864 \n",
       "L 199.894554 93.437781 \n",
       "L 201.488839 93.527402 \n",
       "L 202.285982 93.428326 \n",
       "L 203.083125 93.577242 \n",
       "L 205.474554 93.529298 \n",
       "L 207.068839 93.241591 \n",
       "L 209.460268 93.448716 \n",
       "L 211.851696 93.191952 \n",
       "L 213.445982 93.33211 \n",
       "L 214.243125 93.272319 \n",
       "L 215.040268 93.381051 \n",
       "L 222.214554 93.116312 \n",
       "L 223.011696 93.076129 \n",
       "L 223.808839 93.187225 \n",
       "L 225.403125 93.256269 \n",
       "L 225.403125 93.256269 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_23\">\n",
       "    <path d=\"M 30.103125 139.435158 \n",
       "L 34.088839 139.463598 \n",
       "L 38.074554 135.430731 \n",
       "L 42.060268 122.70075 \n",
       "L 46.045982 121.506247 \n",
       "L 50.031696 115.573553 \n",
       "L 54.017411 111.995734 \n",
       "L 58.003125 110.829672 \n",
       "L 61.988839 107.695526 \n",
       "L 65.974554 107.433873 \n",
       "L 69.960268 105.090373 \n",
       "L 73.945982 103.907247 \n",
       "L 77.931696 102.991462 \n",
       "L 81.917411 102.604671 \n",
       "L 85.903125 102.854948 \n",
       "L 89.888839 100.403374 \n",
       "L 93.874554 100.579705 \n",
       "L 97.860268 98.58318 \n",
       "L 101.845982 99.157679 \n",
       "L 105.831696 99.66392 \n",
       "L 109.817411 98.298775 \n",
       "L 113.803125 98.600244 \n",
       "L 117.788839 97.968864 \n",
       "L 121.774554 96.54115 \n",
       "L 125.760268 97.769781 \n",
       "L 129.745982 96.723169 \n",
       "L 133.731696 97.542256 \n",
       "L 137.717411 96.819867 \n",
       "L 141.703125 95.869953 \n",
       "L 145.688839 96.529774 \n",
       "L 149.674554 96.535462 \n",
       "L 153.660268 95.642429 \n",
       "L 157.645982 95.522979 \n",
       "L 161.631696 96.097478 \n",
       "L 165.617411 96.364819 \n",
       "L 169.603125 96.381883 \n",
       "L 173.588839 95.915458 \n",
       "L 177.574554 96.433076 \n",
       "L 181.560268 95.164628 \n",
       "L 185.545982 95.352336 \n",
       "L 189.531696 95.136188 \n",
       "L 193.517411 96.165735 \n",
       "L 197.503125 95.011049 \n",
       "L 201.488839 96.182799 \n",
       "L 205.474554 95.107747 \n",
       "L 209.460268 96.279497 \n",
       "L 213.445982 95.176004 \n",
       "L 217.431696 94.846094 \n",
       "L 221.417411 94.971233 \n",
       "L 225.403125 94.914351 \n",
       "\" clip-path=\"url(#pd24423e157)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 30.103125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 30.103125 7.2 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 140.634375 59.234375 \n",
       "L 218.403125 59.234375 \n",
       "Q 220.403125 59.234375 220.403125 57.234375 \n",
       "L 220.403125 14.2 \n",
       "Q 220.403125 12.2 218.403125 12.2 \n",
       "L 140.634375 12.2 \n",
       "Q 138.634375 12.2 138.634375 14.2 \n",
       "L 138.634375 57.234375 \n",
       "Q 138.634375 59.234375 140.634375 59.234375 \n",
       "z\n",
       "\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_24\">\n",
       "     <path d=\"M 142.634375 20.298438 \n",
       "L 152.634375 20.298438 \n",
       "L 162.634375 20.298438 \n",
       "\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- train loss -->\n",
       "     <g transform=\"translate(170.634375 23.798438) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_25\">\n",
       "     <path d=\"M 142.634375 34.976563 \n",
       "L 152.634375 34.976563 \n",
       "L 162.634375 34.976563 \n",
       "\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- train acc -->\n",
       "     <g transform=\"translate(170.634375 38.476563) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_26\">\n",
       "     <path d=\"M 142.634375 49.654688 \n",
       "L 152.634375 49.654688 \n",
       "L 162.634375 49.654688 \n",
       "\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- test acc -->\n",
       "     <g transform=\"translate(170.634375 53.154688) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pd24423e157\">\n",
       "   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义神经网络结构\n",
    "# 创建一个序列化的神经网络模型\n",
    "net = nn.Sequential(\n",
    "    # 第一个二维卷积层。输入通道为1（例如，用于灰度图像），输出通道为10。使用5x5的卷积核，边缘填充2个像素。\n",
    "    nn.Conv2d(1, 10, kernel_size=5, padding=2), \n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # 最大池化层，用于降低特征维度。使用2x2的窗口大小和步长2，减少数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第二个二维卷积层。输入通道为10（上一卷积层的输出通道数），输出通道为16。使用5x5的卷积核。\n",
    "    nn.Conv2d(10, 16, kernel_size=5), \n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # 又一个最大池化层，进一步减小数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第三个二维卷积层。输入通道为16（上一卷积层的输出通道数），输出通道为120。使用5x5的卷积核。\n",
    "    nn.Conv2d(16, 120, kernel_size = 5),\n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # Flatten层，将多维度的数据展平成一维，以便传递给全连接层。\n",
    "    nn.Flatten(),\n",
    "    # 全连接（线性）层。输入特征数量为16*5*5（从上一层的输出尺寸计算而来），输出特征数量为120。\n",
    "    nn.Linear(120, 120), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第二个全连接（线性）层，进一步转换特征。输入特征数量为120，输出为84。\n",
    "    nn.Linear(120, 84), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第三个全连接（线性）层，进一步转换特征。输入特征数量为84，输出为20。\n",
    "    nn.Linear(84, 20),\n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第四个全连接（线性）层，最终输出特征数量为10，通常用于10类分类问题。\n",
    "    nn.Linear(20, 10))\n",
    "\n",
    "# 创建一个随机输入张量\n",
    "X = torch.rand(size=(1, 1, 28, 28), dtype=torch.float32)\n",
    "\n",
    "# 遍历改进的LeNet模型的各层并输出各层的输出形状\n",
    "for layer in net:\n",
    "    X = layer(X)\n",
    "    print(layer.__class__.__name__,'output shape: \\t',X.shape)\n",
    "\n",
    "# 加载Fashion MNIST数据集\n",
    "batch_size = 256\n",
    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size=batch_size)\n",
    "\n",
    "# 定义用于在GPU上评估模型精度的函数\n",
    "def evaluate_accuracy_gpu(net, data_iter, device=None): #@save\n",
    "    \"\"\"使用GPU计算模型在数据集上的精度\"\"\"\n",
    "    if isinstance(net, nn.Module):\n",
    "        net.eval()  # 设置为评估模式\n",
    "        if not device:\n",
    "            device = next(iter(net.parameters())).device\n",
    "    # 正确预测的数量，总预测的数量\n",
    "    metric = d2l.Accumulator(2)\n",
    "    with torch.no_grad():\n",
    "        for X, y in data_iter:\n",
    "            if isinstance(X, list):\n",
    "                # BERT微调所需的（之后将介绍）\n",
    "                X = [x.to(device) for x in X]\n",
    "            else:\n",
    "                X = X.to(device)\n",
    "            y = y.to(device)\n",
    "            metric.add(d2l.accuracy(net(X), y), y.numel())\n",
    "    return metric[0] / metric[1]\n",
    "\n",
    "# 定义用于训练模型的函数@save\n",
    "def train_ch6(net, train_iter, test_iter, num_epochs, lr, device):\n",
    "    \"\"\"用GPU训练模型(在第六章定义)\"\"\"\n",
    "    # 定义初始化权重的函数，使用Xavier初始化\n",
    "    def init_weights(m):\n",
    "        if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
    "            nn.init.xavier_uniform_(m.weight)\n",
    "    # 对模型的所有层应用初始化权重的函数\n",
    "    net.apply(init_weights)\n",
    "    # 打印在哪个设备上进行训练（CPU或GPU）\n",
    "    print('training on', device)\n",
    "    # 将模型移动到指定的设备（CPU或GPU）\n",
    "    net.to(device)\n",
    "    # 定义优化器，使用随机梯度下降（SGD）优化器\n",
    "    optimizer = torch.optim.SGD(net.parameters(), lr=lr)\n",
    "    # 定义损失函数，使用交叉熵损失\n",
    "    loss = nn.CrossEntropyLoss()\n",
    "    # 创建用于绘制训练过程的动画\n",
    "    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n",
    "                            legend=['train loss', 'train acc', 'test acc'])\n",
    "    # 创建计时器和获取每个epoch的批量数\n",
    "    timer, num_batches = d2l.Timer(), len(train_iter)\n",
    "    for epoch in range(num_epochs):\n",
    "        # 训练损失之和，训练准确率之和，样本数\n",
    "        metric = d2l.Accumulator(3)\n",
    "        # 设置模型为训练模式\n",
    "        net.train()\n",
    "        for i, (X, y) in enumerate(train_iter):\n",
    "            timer.start()\n",
    "            optimizer.zero_grad()\n",
    "            # 将输入数据和标签移动到指定的设备上\n",
    "            X, y = X.to(device), y.to(device)\n",
    "            # 前向传播\n",
    "            y_hat = net(X)\n",
    "            # 计算损失\n",
    "            l = loss(y_hat, y)\n",
    "            # 反向传播\n",
    "            l.backward()\n",
    "            # 更新模型参数\n",
    "            optimizer.step()\n",
    "            with torch.no_grad():\n",
    "                # 记录训练损失、训练准确率和样本数\n",
    "                metric.add(l * X.shape[0], d2l.accuracy(y_hat, y), X.shape[0])\n",
    "            timer.stop()\n",
    "            train_l = metric[0] / metric[2]\n",
    "            train_acc = metric[1] / metric[2]\n",
    "            # 每完成一定批量数的训练，更新动画\n",
    "            if (i + 1) % (num_batches // 5) == 0 or i == num_batches - 1:\n",
    "                animator.add(epoch + (i + 1) / num_batches,\n",
    "                            (train_l, train_acc, None))\n",
    "        # 计算并记录测试集上的准确率\n",
    "        test_acc = evaluate_accuracy_gpu(net, test_iter)\n",
    "        animator.add(epoch + 1, (None, None, test_acc))\n",
    "    # 打印最终的训练损失、训练准确率和测试准确率\n",
    "    print(f'loss {train_l:.3f}, train acc {train_acc:.3f}, '\n",
    "        f'test acc {test_acc:.3f}')\n",
    "    # 打印训练速度（样本/秒）\n",
    "    print(f'{metric[2] * num_epochs / timer.sum():.1f} examples/sec '\n",
    "        f'on {str(device)}')\n",
    "# 设置学习率和训练周期\n",
    "lr, num_epochs = 0.1, 50\n",
    "\n",
    "# 调用train_ch6函数来在GPU上训练模型，使用你之前定义的函数\n",
    "train_ch6(net, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cek7LSVZ5PSb"
   },
   "source": [
    "&emsp;&emsp; 改进之后的网络在训练集和测试集上均表现出了更好的效果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iDIBoQa0H6Gu"
   },
   "source": [
    "### 练习 6.6.4\n",
    "\n",
    "显示不同输入（例如毛衣和外套）时，`LeNet`第一层和第二层的激活值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Mq9U1KWRH6Gu"
   },
   "source": [
    "**解答：** "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TK6125gm5eK_"
   },
   "source": [
    "&emsp;&emsp; 通过在所有`epoch`结束后添加`d2l.show_images`展示了激活值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "346c706c",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss 0.246, train acc 0.911, test acc 0.880\n",
      "107781.6 examples/sec on cuda:0\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"238.965625pt\" height=\"183.35625pt\" viewBox=\"0 0 238.965625 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-03-01T02:17:43.115076</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 183.35625 \n",
       "L 238.965625 183.35625 \n",
       "L 238.965625 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "L 30.103125 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_1\">\n",
       "      <path d=\"M 65.974554 145.8 \n",
       "L 65.974554 7.2 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path id=\"m07fbb1be9a\" d=\"M 0 0 \n",
       "L 0 3.5 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m07fbb1be9a\" x=\"65.974554\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 10 -->\n",
       "      <g transform=\"translate(59.612054 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-31\" d=\"M 794 531 \n",
       "L 1825 531 \n",
       "L 1825 4091 \n",
       "L 703 3866 \n",
       "L 703 4441 \n",
       "L 1819 4666 \n",
       "L 2450 4666 \n",
       "L 2450 531 \n",
       "L 3481 531 \n",
       "L 3481 0 \n",
       "L 794 0 \n",
       "L 794 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \n",
       "Q 1547 4250 1301 3770 \n",
       "Q 1056 3291 1056 2328 \n",
       "Q 1056 1369 1301 889 \n",
       "Q 1547 409 2034 409 \n",
       "Q 2525 409 2770 889 \n",
       "Q 3016 1369 3016 2328 \n",
       "Q 3016 3291 2770 3770 \n",
       "Q 2525 4250 2034 4250 \n",
       "z\n",
       "M 2034 4750 \n",
       "Q 2819 4750 3233 4129 \n",
       "Q 3647 3509 3647 2328 \n",
       "Q 3647 1150 3233 529 \n",
       "Q 2819 -91 2034 -91 \n",
       "Q 1250 -91 836 529 \n",
       "Q 422 1150 422 2328 \n",
       "Q 422 3509 836 4129 \n",
       "Q 1250 4750 2034 4750 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_3\">\n",
       "      <path d=\"M 105.831696 145.8 \n",
       "L 105.831696 7.2 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m07fbb1be9a\" x=\"105.831696\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 20 -->\n",
       "      <g transform=\"translate(99.469196 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \n",
       "L 3431 531 \n",
       "L 3431 0 \n",
       "L 469 0 \n",
       "L 469 531 \n",
       "Q 828 903 1448 1529 \n",
       "Q 2069 2156 2228 2338 \n",
       "Q 2531 2678 2651 2914 \n",
       "Q 2772 3150 2772 3378 \n",
       "Q 2772 3750 2511 3984 \n",
       "Q 2250 4219 1831 4219 \n",
       "Q 1534 4219 1204 4116 \n",
       "Q 875 4013 500 3803 \n",
       "L 500 4441 \n",
       "Q 881 4594 1212 4672 \n",
       "Q 1544 4750 1819 4750 \n",
       "Q 2544 4750 2975 4387 \n",
       "Q 3406 4025 3406 3419 \n",
       "Q 3406 3131 3298 2873 \n",
       "Q 3191 2616 2906 2266 \n",
       "Q 2828 2175 2409 1742 \n",
       "Q 1991 1309 1228 531 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path d=\"M 145.688839 145.8 \n",
       "L 145.688839 7.2 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m07fbb1be9a\" x=\"145.688839\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 30 -->\n",
       "      <g transform=\"translate(139.326339 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \n",
       "Q 3050 2419 3304 2112 \n",
       "Q 3559 1806 3559 1356 \n",
       "Q 3559 666 3084 287 \n",
       "Q 2609 -91 1734 -91 \n",
       "Q 1441 -91 1130 -33 \n",
       "Q 819 25 488 141 \n",
       "L 488 750 \n",
       "Q 750 597 1062 519 \n",
       "Q 1375 441 1716 441 \n",
       "Q 2309 441 2620 675 \n",
       "Q 2931 909 2931 1356 \n",
       "Q 2931 1769 2642 2001 \n",
       "Q 2353 2234 1838 2234 \n",
       "L 1294 2234 \n",
       "L 1294 2753 \n",
       "L 1863 2753 \n",
       "Q 2328 2753 2575 2939 \n",
       "Q 2822 3125 2822 3475 \n",
       "Q 2822 3834 2567 4026 \n",
       "Q 2313 4219 1838 4219 \n",
       "Q 1578 4219 1281 4162 \n",
       "Q 984 4106 628 3988 \n",
       "L 628 4550 \n",
       "Q 988 4650 1302 4700 \n",
       "Q 1616 4750 1894 4750 \n",
       "Q 2613 4750 3031 4423 \n",
       "Q 3450 4097 3450 3541 \n",
       "Q 3450 3153 3228 2886 \n",
       "Q 3006 2619 2597 2516 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-33\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <path d=\"M 185.545982 145.8 \n",
       "L 185.545982 7.2 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m07fbb1be9a\" x=\"185.545982\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 40 -->\n",
       "      <g transform=\"translate(179.183482 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \n",
       "L 825 1625 \n",
       "L 2419 1625 \n",
       "L 2419 4116 \n",
       "z\n",
       "M 2253 4666 \n",
       "L 3047 4666 \n",
       "L 3047 1625 \n",
       "L 3713 1625 \n",
       "L 3713 1100 \n",
       "L 3047 1100 \n",
       "L 3047 0 \n",
       "L 2419 0 \n",
       "L 2419 1100 \n",
       "L 313 1100 \n",
       "L 313 1709 \n",
       "L 2253 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-34\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m07fbb1be9a\" x=\"225.403125\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 50 -->\n",
       "      <g transform=\"translate(219.040625 160.398438) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \n",
       "L 3169 4666 \n",
       "L 3169 4134 \n",
       "L 1269 4134 \n",
       "L 1269 2991 \n",
       "Q 1406 3038 1543 3061 \n",
       "Q 1681 3084 1819 3084 \n",
       "Q 2600 3084 3056 2656 \n",
       "Q 3513 2228 3513 1497 \n",
       "Q 3513 744 3044 326 \n",
       "Q 2575 -91 1722 -91 \n",
       "Q 1428 -91 1123 -41 \n",
       "Q 819 9 494 109 \n",
       "L 494 744 \n",
       "Q 775 591 1075 516 \n",
       "Q 1375 441 1709 441 \n",
       "Q 2250 441 2565 725 \n",
       "Q 2881 1009 2881 1497 \n",
       "Q 2881 1984 2565 2268 \n",
       "Q 2250 2553 1709 2553 \n",
       "Q 1456 2553 1204 2497 \n",
       "Q 953 2441 691 2322 \n",
       "L 691 4666 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-35\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <g transform=\"translate(112.525 174.076563) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \n",
       "L 3597 1613 \n",
       "L 953 1613 \n",
       "Q 991 1019 1311 708 \n",
       "Q 1631 397 2203 397 \n",
       "Q 2534 397 2845 478 \n",
       "Q 3156 559 3463 722 \n",
       "L 3463 178 \n",
       "Q 3153 47 2828 -22 \n",
       "Q 2503 -91 2169 -91 \n",
       "Q 1331 -91 842 396 \n",
       "Q 353 884 353 1716 \n",
       "Q 353 2575 817 3079 \n",
       "Q 1281 3584 2069 3584 \n",
       "Q 2775 3584 3186 3129 \n",
       "Q 3597 2675 3597 1894 \n",
       "z\n",
       "M 3022 2063 \n",
       "Q 3016 2534 2758 2815 \n",
       "Q 2500 3097 2075 3097 \n",
       "Q 1594 3097 1305 2825 \n",
       "Q 1016 2553 972 2059 \n",
       "L 3022 2063 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \n",
       "L 1159 -1331 \n",
       "L 581 -1331 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2969 \n",
       "Q 1341 3281 1617 3432 \n",
       "Q 1894 3584 2278 3584 \n",
       "Q 2916 3584 3314 3078 \n",
       "Q 3713 2572 3713 1747 \n",
       "Q 3713 922 3314 415 \n",
       "Q 2916 -91 2278 -91 \n",
       "Q 1894 -91 1617 61 \n",
       "Q 1341 213 1159 525 \n",
       "z\n",
       "M 3116 1747 \n",
       "Q 3116 2381 2855 2742 \n",
       "Q 2594 3103 2138 3103 \n",
       "Q 1681 3103 1420 2742 \n",
       "Q 1159 2381 1159 1747 \n",
       "Q 1159 1113 1420 752 \n",
       "Q 1681 391 2138 391 \n",
       "Q 2594 391 2855 752 \n",
       "Q 3116 1113 3116 1747 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \n",
       "Q 1497 3097 1228 2736 \n",
       "Q 959 2375 959 1747 \n",
       "Q 959 1119 1226 758 \n",
       "Q 1494 397 1959 397 \n",
       "Q 2419 397 2687 759 \n",
       "Q 2956 1122 2956 1747 \n",
       "Q 2956 2369 2687 2733 \n",
       "Q 2419 3097 1959 3097 \n",
       "z\n",
       "M 1959 3584 \n",
       "Q 2709 3584 3137 3096 \n",
       "Q 3566 2609 3566 1747 \n",
       "Q 3566 888 3137 398 \n",
       "Q 2709 -91 1959 -91 \n",
       "Q 1206 -91 779 398 \n",
       "Q 353 888 353 1747 \n",
       "Q 353 2609 779 3096 \n",
       "Q 1206 3584 1959 3584 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \n",
       "L 3122 2828 \n",
       "Q 2878 2963 2633 3030 \n",
       "Q 2388 3097 2138 3097 \n",
       "Q 1578 3097 1268 2742 \n",
       "Q 959 2388 959 1747 \n",
       "Q 959 1106 1268 751 \n",
       "Q 1578 397 2138 397 \n",
       "Q 2388 397 2633 464 \n",
       "Q 2878 531 3122 666 \n",
       "L 3122 134 \n",
       "Q 2881 22 2623 -34 \n",
       "Q 2366 -91 2075 -91 \n",
       "Q 1284 -91 818 406 \n",
       "Q 353 903 353 1747 \n",
       "Q 353 2603 823 3093 \n",
       "Q 1294 3584 2113 3584 \n",
       "Q 2378 3584 2631 3529 \n",
       "Q 2884 3475 3122 3366 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 4863 \n",
       "L 1159 4863 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-65\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path d=\"M 30.103125 144.709089 \n",
       "L 225.403125 144.709089 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <defs>\n",
       "       <path id=\"m4afc8becd6\" d=\"M 0 0 \n",
       "L -3.5 0 \n",
       "\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use xlink:href=\"#m4afc8becd6\" x=\"30.103125\" y=\"144.709089\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.0 -->\n",
       "      <g transform=\"translate(7.2 148.508308) scale(0.1 -0.1)\">\n",
       "       <defs>\n",
       "        <path id=\"DejaVuSans-2e\" d=\"M 684 794 \n",
       "L 1344 794 \n",
       "L 1344 0 \n",
       "L 684 0 \n",
       "L 684 794 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       </defs>\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <path d=\"M 30.103125 116.552862 \n",
       "L 225.403125 116.552862 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m4afc8becd6\" x=\"30.103125\" y=\"116.552862\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.5 -->\n",
       "      <g transform=\"translate(7.2 120.35208) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-30\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <path d=\"M 30.103125 88.396634 \n",
       "L 225.403125 88.396634 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m4afc8becd6\" x=\"30.103125\" y=\"88.396634\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(7.2 92.195853) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path d=\"M 30.103125 60.240406 \n",
       "L 225.403125 60.240406 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m4afc8becd6\" x=\"30.103125\" y=\"60.240406\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(7.2 64.039625) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-31\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-35\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <path d=\"M 30.103125 32.084179 \n",
       "L 225.403125 32.084179 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use xlink:href=\"#m4afc8becd6\" x=\"30.103125\" y=\"32.084179\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 2.0 -->\n",
       "      <g transform=\"translate(7.2 35.883398) scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#DejaVuSans-32\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-2e\" x=\"63.623047\"/>\n",
       "       <use xlink:href=\"#DejaVuSans-30\" x=\"95.410156\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"line2d_21\">\n",
       "    <path d=\"M 26.914554 13.5 \n",
       "L 27.711696 14.245445 \n",
       "L 28.508839 14.501341 \n",
       "L 30.103125 14.702714 \n",
       "L 30.900268 15.017109 \n",
       "L 42.060268 15.100866 \n",
       "L 46.045982 15.632043 \n",
       "L 47.640268 22.181323 \n",
       "L 50.031696 34.034895 \n",
       "L 50.828839 53.66383 \n",
       "L 53.220268 59.51344 \n",
       "L 54.017411 61.244605 \n",
       "L 54.814554 70.015722 \n",
       "L 55.611696 71.236275 \n",
       "L 56.408839 72.203216 \n",
       "L 58.003125 73.681724 \n",
       "L 58.800268 78.552815 \n",
       "L 59.597411 79.091966 \n",
       "L 60.394554 79.905785 \n",
       "L 61.191696 80.388368 \n",
       "L 61.988839 81.088895 \n",
       "L 62.785982 84.99784 \n",
       "L 63.583125 85.565001 \n",
       "L 64.380268 86.330658 \n",
       "L 65.974554 87.176767 \n",
       "L 66.771696 90.647841 \n",
       "L 67.568839 90.721632 \n",
       "L 69.163125 91.727186 \n",
       "L 69.960268 92.156575 \n",
       "L 70.757411 94.098433 \n",
       "L 71.554554 94.853521 \n",
       "L 72.351696 94.052207 \n",
       "L 73.945982 95.31119 \n",
       "L 74.743125 97.977072 \n",
       "L 75.540268 98.612541 \n",
       "L 76.337411 98.810156 \n",
       "L 77.931696 99.536254 \n",
       "L 78.728839 101.889833 \n",
       "L 79.525982 102.29514 \n",
       "L 80.323125 102.279805 \n",
       "L 81.917411 103.081919 \n",
       "L 83.511696 104.940654 \n",
       "L 85.105982 106.217132 \n",
       "L 85.903125 106.49396 \n",
       "L 86.700268 108.774346 \n",
       "L 89.888839 110.166187 \n",
       "L 90.685982 112.903491 \n",
       "L 91.483125 112.483706 \n",
       "L 92.280268 112.501569 \n",
       "L 93.874554 112.877603 \n",
       "L 94.671696 114.513506 \n",
       "L 96.265982 114.711603 \n",
       "L 97.063125 114.583079 \n",
       "L 97.860268 114.73709 \n",
       "L 98.657411 115.961281 \n",
       "L 101.845982 116.280459 \n",
       "L 102.643125 117.76042 \n",
       "L 103.440268 117.524655 \n",
       "L 105.034554 117.748123 \n",
       "L 105.831696 117.755328 \n",
       "L 106.628839 118.868623 \n",
       "L 107.425982 118.653144 \n",
       "L 108.223125 118.84104 \n",
       "L 109.817411 118.856088 \n",
       "L 112.208839 119.398724 \n",
       "L 113.803125 119.591523 \n",
       "L 114.600268 120.417049 \n",
       "L 115.397411 120.672692 \n",
       "L 116.991696 120.607198 \n",
       "L 117.788839 120.646557 \n",
       "L 118.585982 121.423073 \n",
       "L 120.180268 121.302516 \n",
       "L 120.977411 121.198506 \n",
       "L 121.774554 121.59146 \n",
       "L 122.571696 122.440729 \n",
       "L 123.368839 122.097825 \n",
       "L 125.760268 122.29009 \n",
       "L 126.557411 122.88399 \n",
       "L 127.354554 122.641418 \n",
       "L 128.948839 122.734887 \n",
       "L 130.543125 123.068279 \n",
       "L 131.340268 123.378287 \n",
       "L 132.137411 123.200821 \n",
       "L 133.731696 123.384544 \n",
       "L 134.528839 124.043117 \n",
       "L 137.717411 124.128367 \n",
       "L 138.514554 124.997244 \n",
       "L 140.108839 124.55875 \n",
       "L 141.703125 124.804714 \n",
       "L 142.500268 125.525251 \n",
       "L 143.297411 124.96862 \n",
       "L 144.891696 125.263313 \n",
       "L 145.688839 125.318153 \n",
       "L 146.485982 125.220901 \n",
       "L 147.283125 125.637855 \n",
       "L 148.080268 125.551969 \n",
       "L 149.674554 125.716597 \n",
       "L 150.471696 125.488873 \n",
       "L 151.268839 125.577167 \n",
       "L 152.065982 125.897443 \n",
       "L 152.863125 126.079634 \n",
       "L 153.660268 126.064215 \n",
       "L 155.254554 126.463609 \n",
       "L 156.848839 126.56158 \n",
       "L 157.645982 126.458061 \n",
       "L 158.443125 126.817708 \n",
       "L 160.037411 126.919033 \n",
       "L 161.631696 126.972901 \n",
       "L 162.428839 127.462907 \n",
       "L 163.225982 127.287743 \n",
       "L 165.617411 127.248666 \n",
       "L 166.414554 127.884119 \n",
       "L 167.211696 127.770772 \n",
       "L 168.008839 127.39007 \n",
       "L 169.603125 127.493079 \n",
       "L 171.197411 127.89292 \n",
       "L 175.183125 127.647128 \n",
       "L 175.980268 127.923527 \n",
       "L 177.574554 128.086843 \n",
       "L 178.371696 129.230279 \n",
       "L 179.168839 128.608408 \n",
       "L 180.763125 128.64615 \n",
       "L 181.560268 128.482123 \n",
       "L 182.357411 128.530849 \n",
       "L 183.154554 128.823463 \n",
       "L 185.545982 128.702303 \n",
       "L 186.343125 128.86165 \n",
       "L 187.140268 128.616132 \n",
       "L 187.937411 128.851741 \n",
       "L 188.734554 128.942273 \n",
       "L 189.531696 128.894824 \n",
       "L 190.328839 129.846775 \n",
       "L 191.923125 129.559858 \n",
       "L 194.314554 129.148079 \n",
       "L 195.111696 129.383766 \n",
       "L 195.908839 129.281279 \n",
       "L 196.705982 129.417413 \n",
       "L 197.503125 129.365497 \n",
       "L 198.300268 130.181928 \n",
       "L 199.097411 130.249884 \n",
       "L 199.894554 129.721947 \n",
       "L 201.488839 129.59826 \n",
       "L 202.285982 130.078942 \n",
       "L 203.083125 130.124152 \n",
       "L 204.677411 129.734629 \n",
       "L 205.474554 129.864702 \n",
       "L 206.271696 129.819254 \n",
       "L 207.068839 130.078746 \n",
       "L 209.460268 130.077342 \n",
       "L 210.257411 130.360837 \n",
       "L 213.445982 130.354024 \n",
       "L 214.243125 130.705275 \n",
       "L 219.025982 130.209965 \n",
       "L 219.823125 130.562276 \n",
       "L 220.620268 130.657368 \n",
       "L 221.417411 130.616015 \n",
       "L 222.214554 130.69185 \n",
       "L 223.011696 131.051387 \n",
       "L 223.808839 130.935567 \n",
       "L 224.605982 130.992838 \n",
       "L 225.403125 130.880636 \n",
       "L 225.403125 130.880636 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_22\">\n",
       "    <path d=\"M 26.914554 138.929013 \n",
       "L 30.103125 139.158558 \n",
       "L 30.900268 139.5 \n",
       "L 31.697411 139.193445 \n",
       "L 34.088839 139.121955 \n",
       "L 34.885982 138.774565 \n",
       "L 36.480268 138.960214 \n",
       "L 38.074554 138.922046 \n",
       "L 38.871696 138.961774 \n",
       "L 39.668839 138.699682 \n",
       "L 40.465982 138.651319 \n",
       "L 41.263125 138.482051 \n",
       "L 42.060268 138.512842 \n",
       "L 42.857411 138.058491 \n",
       "L 43.654554 138.194218 \n",
       "L 44.451696 137.871282 \n",
       "L 45.248839 137.381029 \n",
       "L 46.045982 136.451806 \n",
       "L 46.843125 128.83845 \n",
       "L 47.640268 129.030339 \n",
       "L 48.437411 128.769806 \n",
       "L 50.031696 126.944387 \n",
       "L 50.828839 121.153522 \n",
       "L 51.625982 120.748683 \n",
       "L 52.423125 120.181595 \n",
       "L 54.017411 119.642538 \n",
       "L 54.814554 118.233062 \n",
       "L 56.408839 117.506067 \n",
       "L 58.003125 116.429913 \n",
       "L 58.800268 113.721326 \n",
       "L 60.394554 112.995891 \n",
       "L 61.988839 112.664487 \n",
       "L 62.785982 111.282929 \n",
       "L 63.583125 110.562175 \n",
       "L 65.177411 110.344545 \n",
       "L 65.974554 110.295609 \n",
       "L 66.771696 109.467003 \n",
       "L 67.568839 109.335956 \n",
       "L 69.163125 108.848043 \n",
       "L 69.960268 108.605297 \n",
       "L 70.757411 107.496628 \n",
       "L 71.554554 107.293039 \n",
       "L 72.351696 107.557471 \n",
       "L 73.945982 107.173084 \n",
       "L 74.743125 106.050439 \n",
       "L 75.540268 105.666661 \n",
       "L 76.337411 105.549655 \n",
       "L 77.931696 105.164606 \n",
       "L 78.728839 104.094105 \n",
       "L 79.525982 103.857754 \n",
       "L 80.323125 103.814852 \n",
       "L 81.917411 103.374809 \n",
       "L 82.714554 103.003613 \n",
       "L 83.511696 102.493469 \n",
       "L 85.903125 102.15846 \n",
       "L 86.700268 101.590186 \n",
       "L 89.091696 101.27427 \n",
       "L 89.888839 101.202086 \n",
       "L 90.685982 100.21888 \n",
       "L 91.483125 100.375667 \n",
       "L 93.874554 100.238205 \n",
       "L 94.671696 99.526207 \n",
       "L 97.860268 99.445138 \n",
       "L 98.657411 98.922458 \n",
       "L 99.454554 98.770351 \n",
       "L 100.251696 98.76489 \n",
       "L 101.845982 98.617345 \n",
       "L 102.643125 97.939611 \n",
       "L 105.831696 97.888098 \n",
       "L 106.628839 97.317141 \n",
       "L 107.425982 97.485629 \n",
       "L 109.020268 97.439997 \n",
       "L 110.614554 97.363943 \n",
       "L 113.803125 97.220796 \n",
       "L 115.397411 96.860819 \n",
       "L 117.788839 96.876351 \n",
       "L 118.585982 96.605747 \n",
       "L 119.383125 96.523843 \n",
       "L 120.977411 96.620958 \n",
       "L 122.571696 96.203248 \n",
       "L 123.368839 96.411518 \n",
       "L 126.557411 96.067521 \n",
       "L 128.151696 96.151765 \n",
       "L 130.543125 95.987957 \n",
       "L 131.340268 95.854571 \n",
       "L 132.934554 95.966896 \n",
       "L 133.731696 95.954704 \n",
       "L 134.528839 95.707144 \n",
       "L 137.717411 95.705052 \n",
       "L 138.514554 95.379528 \n",
       "L 140.108839 95.607299 \n",
       "L 141.703125 95.50702 \n",
       "L 142.500268 95.356127 \n",
       "L 143.297411 95.508234 \n",
       "L 144.891696 95.338576 \n",
       "L 145.688839 95.319312 \n",
       "L 146.485982 95.445051 \n",
       "L 147.283125 95.218061 \n",
       "L 148.877411 95.216891 \n",
       "L 149.674554 95.190732 \n",
       "L 151.268839 95.283584 \n",
       "L 152.863125 95.091695 \n",
       "L 153.660268 95.104386 \n",
       "L 155.254554 94.855343 \n",
       "L 158.443125 94.860023 \n",
       "L 160.037411 94.727417 \n",
       "L 165.617411 94.661395 \n",
       "L 166.414554 94.36392 \n",
       "L 168.805982 94.616652 \n",
       "L 169.603125 94.605082 \n",
       "L 171.197411 94.333498 \n",
       "L 175.183125 94.57687 \n",
       "L 175.980268 94.410722 \n",
       "L 177.574554 94.380771 \n",
       "L 178.371696 93.905258 \n",
       "L 179.168839 94.15565 \n",
       "L 182.357411 94.232873 \n",
       "L 183.154554 94.026944 \n",
       "L 187.140268 94.305417 \n",
       "L 188.734554 94.071406 \n",
       "L 189.531696 94.107656 \n",
       "L 190.328839 93.872496 \n",
       "L 197.503125 93.904931 \n",
       "L 199.097411 93.589343 \n",
       "L 199.894554 93.817894 \n",
       "L 201.488839 93.843926 \n",
       "L 202.285982 93.572962 \n",
       "L 203.880268 93.688407 \n",
       "L 204.677411 93.780062 \n",
       "L 207.865982 93.597923 \n",
       "L 209.460268 93.640262 \n",
       "L 211.054554 93.626784 \n",
       "L 212.648839 93.505099 \n",
       "L 213.445982 93.588642 \n",
       "L 215.837411 93.510559 \n",
       "L 219.025982 93.594023 \n",
       "L 219.823125 93.496518 \n",
       "L 221.417411 93.523883 \n",
       "L 223.011696 93.369372 \n",
       "L 225.403125 93.39061 \n",
       "L 225.403125 93.39061 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_23\">\n",
       "    <path d=\"M 30.103125 139.077844 \n",
       "L 34.088839 139.077844 \n",
       "L 38.074554 139.077844 \n",
       "L 42.060268 139.077844 \n",
       "L 46.045982 132.782111 \n",
       "L 50.031696 122.623344 \n",
       "L 54.017411 116.761218 \n",
       "L 58.003125 117.521436 \n",
       "L 61.988839 111.411535 \n",
       "L 65.974554 111.293278 \n",
       "L 69.960268 109.874205 \n",
       "L 73.945982 108.719799 \n",
       "L 77.931696 104.102178 \n",
       "L 81.917411 104.242959 \n",
       "L 85.903125 103.52216 \n",
       "L 89.888839 100.954312 \n",
       "L 93.874554 100.323612 \n",
       "L 97.860268 101.472386 \n",
       "L 101.845982 98.583557 \n",
       "L 105.831696 100.802268 \n",
       "L 109.817411 98.803176 \n",
       "L 113.803125 97.637508 \n",
       "L 117.788839 98.628607 \n",
       "L 121.774554 97.688189 \n",
       "L 125.760268 98.859488 \n",
       "L 129.745982 100.531968 \n",
       "L 133.731696 97.519252 \n",
       "L 137.717411 96.781559 \n",
       "L 141.703125 96.567571 \n",
       "L 145.688839 96.269115 \n",
       "L 149.674554 96.100178 \n",
       "L 153.660268 96.302903 \n",
       "L 157.645982 95.908716 \n",
       "L 161.631696 95.829878 \n",
       "L 165.617411 96.20154 \n",
       "L 169.603125 95.615891 \n",
       "L 173.588839 96.65204 \n",
       "L 177.574554 95.576472 \n",
       "L 181.560268 96.38174 \n",
       "L 185.545982 96.072022 \n",
       "L 189.531696 95.131604 \n",
       "L 193.517411 95.43006 \n",
       "L 197.503125 95.587735 \n",
       "L 201.488839 95.171022 \n",
       "L 205.474554 95.092185 \n",
       "L 209.460268 95.745409 \n",
       "L 213.445982 96.105809 \n",
       "L 217.431696 96.342322 \n",
       "L 221.417411 95.041504 \n",
       "L 225.403125 95.171022 \n",
       "\" clip-path=\"url(#pf04affbd80)\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 30.103125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 225.403125 145.8 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 30.103125 145.8 \n",
       "L 225.403125 145.8 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 30.103125 7.2 \n",
       "L 225.403125 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 140.634375 59.234375 \n",
       "L 218.403125 59.234375 \n",
       "Q 220.403125 59.234375 220.403125 57.234375 \n",
       "L 220.403125 14.2 \n",
       "Q 220.403125 12.2 218.403125 12.2 \n",
       "L 140.634375 12.2 \n",
       "Q 138.634375 12.2 138.634375 14.2 \n",
       "L 138.634375 57.234375 \n",
       "Q 138.634375 59.234375 140.634375 59.234375 \n",
       "z\n",
       "\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_24\">\n",
       "     <path d=\"M 142.634375 20.298438 \n",
       "L 152.634375 20.298438 \n",
       "L 162.634375 20.298438 \n",
       "\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n",
       "    </g>\n",
       "    <g id=\"text_12\">\n",
       "     <!-- train loss -->\n",
       "     <g transform=\"translate(170.634375 23.798438) scale(0.1 -0.1)\">\n",
       "      <defs>\n",
       "       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \n",
       "L 1172 3500 \n",
       "L 2356 3500 \n",
       "L 2356 3053 \n",
       "L 1172 3053 \n",
       "L 1172 1153 \n",
       "Q 1172 725 1289 603 \n",
       "Q 1406 481 1766 481 \n",
       "L 2356 481 \n",
       "L 2356 0 \n",
       "L 1766 0 \n",
       "Q 1100 0 847 248 \n",
       "Q 594 497 594 1153 \n",
       "L 594 3053 \n",
       "L 172 3053 \n",
       "L 172 3500 \n",
       "L 594 3500 \n",
       "L 594 4494 \n",
       "L 1172 4494 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \n",
       "Q 2534 3019 2420 3045 \n",
       "Q 2306 3072 2169 3072 \n",
       "Q 1681 3072 1420 2755 \n",
       "Q 1159 2438 1159 1844 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1341 3275 1631 3429 \n",
       "Q 1922 3584 2338 3584 \n",
       "Q 2397 3584 2469 3576 \n",
       "Q 2541 3569 2628 3553 \n",
       "L 2631 2963 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \n",
       "Q 1497 1759 1228 1600 \n",
       "Q 959 1441 959 1056 \n",
       "Q 959 750 1161 570 \n",
       "Q 1363 391 1709 391 \n",
       "Q 2188 391 2477 730 \n",
       "Q 2766 1069 2766 1631 \n",
       "L 2766 1759 \n",
       "L 2194 1759 \n",
       "z\n",
       "M 3341 1997 \n",
       "L 3341 0 \n",
       "L 2766 0 \n",
       "L 2766 531 \n",
       "Q 2569 213 2275 61 \n",
       "Q 1981 -91 1556 -91 \n",
       "Q 1019 -91 701 211 \n",
       "Q 384 513 384 1019 \n",
       "Q 384 1609 779 1909 \n",
       "Q 1175 2209 1959 2209 \n",
       "L 2766 2209 \n",
       "L 2766 2266 \n",
       "Q 2766 2663 2505 2880 \n",
       "Q 2244 3097 1772 3097 \n",
       "Q 1472 3097 1187 3025 \n",
       "Q 903 2953 641 2809 \n",
       "L 641 3341 \n",
       "Q 956 3463 1253 3523 \n",
       "Q 1550 3584 1831 3584 \n",
       "Q 2591 3584 2966 3190 \n",
       "Q 3341 2797 3341 1997 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \n",
       "L 1178 3500 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 3500 \n",
       "z\n",
       "M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 4134 \n",
       "L 603 4134 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \n",
       "L 3513 0 \n",
       "L 2938 0 \n",
       "L 2938 2094 \n",
       "Q 2938 2591 2744 2837 \n",
       "Q 2550 3084 2163 3084 \n",
       "Q 1697 3084 1428 2787 \n",
       "Q 1159 2491 1159 1978 \n",
       "L 1159 0 \n",
       "L 581 0 \n",
       "L 581 3500 \n",
       "L 1159 3500 \n",
       "L 1159 2956 \n",
       "Q 1366 3272 1645 3428 \n",
       "Q 1925 3584 2291 3584 \n",
       "Q 2894 3584 3203 3211 \n",
       "Q 3513 2838 3513 2113 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \n",
       "L 1178 4863 \n",
       "L 1178 0 \n",
       "L 603 0 \n",
       "L 603 4863 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \n",
       "L 2834 2853 \n",
       "Q 2591 2978 2328 3040 \n",
       "Q 2066 3103 1784 3103 \n",
       "Q 1356 3103 1142 2972 \n",
       "Q 928 2841 928 2578 \n",
       "Q 928 2378 1081 2264 \n",
       "Q 1234 2150 1697 2047 \n",
       "L 1894 2003 \n",
       "Q 2506 1872 2764 1633 \n",
       "Q 3022 1394 3022 966 \n",
       "Q 3022 478 2636 193 \n",
       "Q 2250 -91 1575 -91 \n",
       "Q 1294 -91 989 -36 \n",
       "Q 684 19 347 128 \n",
       "L 347 722 \n",
       "Q 666 556 975 473 \n",
       "Q 1284 391 1588 391 \n",
       "Q 1994 391 2212 530 \n",
       "Q 2431 669 2431 922 \n",
       "Q 2431 1156 2273 1281 \n",
       "Q 2116 1406 1581 1522 \n",
       "L 1381 1569 \n",
       "Q 847 1681 609 1914 \n",
       "Q 372 2147 372 2553 \n",
       "Q 372 3047 722 3315 \n",
       "Q 1072 3584 1716 3584 \n",
       "Q 2034 3584 2315 3537 \n",
       "Q 2597 3491 2834 3397 \n",
       "z\n",
       "\" transform=\"scale(0.015625)\"/>\n",
       "      </defs>\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_25\">\n",
       "     <path d=\"M 142.634375 34.976563 \n",
       "L 152.634375 34.976563 \n",
       "L 162.634375 34.976563 \n",
       "\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_13\">\n",
       "     <!-- train acc -->\n",
       "     <g transform=\"translate(170.634375 38.476563) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"264.550781\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"325.830078\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"380.810547\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_26\">\n",
       "     <path d=\"M 142.634375 49.654688 \n",
       "L 152.634375 49.654688 \n",
       "L 162.634375 49.654688 \n",
       "\" style=\"fill: none; stroke-dasharray: 9.6,2.4,1.5,2.4; stroke-dashoffset: 0; stroke: #008000; stroke-width: 1.5\"/>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- test acc -->\n",
       "     <g transform=\"translate(170.634375 53.154688) scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#DejaVuSans-74\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-65\" x=\"39.208984\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-73\" x=\"100.732422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-74\" x=\"152.832031\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-20\" x=\"192.041016\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-61\" x=\"223.828125\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"285.107422\"/>\n",
       "      <use xlink:href=\"#DejaVuSans-63\" x=\"340.087891\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pf04affbd80\">\n",
       "   <rect x=\"30.103125\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 350x250 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"767.7pt\" height=\"85.466038pt\" viewBox=\"0 0 767.7 85.466038\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-03-01T02:17:43.181977</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 85.466038 \n",
       "L 767.7 85.466038 \n",
       "L 767.7 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 7.2 78.266038 \n",
       "L 78.266038 78.266038 \n",
       "L 78.266038 7.2 \n",
       "L 7.2 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#pb5a419526e)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAAHEElEQVR4nO2c329URRTH76/91d7dbltKf9Ha0gINibaoVIPESEg0EtHERx990Qf8E/wPkETfDMbEGB+NPogGE0U0wYiJBqOlhIKF1tLSX8t2u91f965vc+Y7eDdthXpMzufpzD17587u986ZmXPnrj14+p26JbDA+a8bIBAiBiNEDEaIGIwQMRghYjBCxGCEiMEIEYMRIgYjRAxGiBiMEDEYIWIwQsRghIjBCBGDEV7QUoMDdjxUdtIvg6+leYPsRAl82QT50h6el3DoGqFlgy/tYT06tZDulZRbBV+17lKddawzZgdQLgQJZTs2PthscirKLoZx8LkW/RaeE4JvpdKs7LVaAnx6e/7MteN5OTqvHmC7pWcwQsRghNfVtwIHFhZblF2ruuC7u5RR9vw6dj+7Zv+jbRKmMISk2im8lUsxrNOhkOIYYaKaS5KvGUNYuI71OBt0zzmdGBa9GLWnXMTzWrJFZaeTGHoX876ygxre0yM9C8rOpjbAt1akkJZpxrZIz2CEiMEIEYMRnvPhLjgw8t1NZVcP7gFfkKAxJL68Dj53eU3Z9QL6gpUcfc5vBp8VpzgdLC1vrtUm449C0bk6BeVwjdrmDfSDrzZ9O7peRxszQxzroBYbx8jK2EFlx1fXwLc3Rr/NnRe68HLRLRF2GhGDEV7L+atwIMjnlR2bS4Ov8ORurZQCn752LY5h9/MK1MU3fJwu+9dWqbCFMOUODyr7zmFsZ5u/H8q1Zm21HsOQ0tQoTBmhKYrguUNQLrdS6M3cXcXPatfrjhtT8E1dTdgRRAxGiBiM8ArHRuBA07lfqLB6D3z+HKVKnIu/gk/PhQZDT4MvpsXeWN6Iw4uYjtk0Lo0D2akKuOLLRSgXerPKrvrgspq2d3XL66NpfyVu3NPaj5Ef7wOXfz6n7FomCT7pGYwQMRjhzR3Fqeb+CVpbBtdvgs+5uLmp58IrmOEMq6S5/xs+iOn+dnur7uAarbLj19AXGp+tHD2i7FK79UCozczS9TXbsnCab6K3zb1yHXzSMxghYjBCxGCE98aLX8OBD/qfUXZlZRx8mUlP2V3vXoqsdPj1CSg7wwPKLvZH6+/uwoC+7SyuHf2kse+bYqSvEd6eXiiHWtvCkvH0sLdH2fVmTBsFU9PKtvt6wCc9gxEiBiO8T29jxrE6T2vSzn1L4GsboS5eP4tr17Co+co4tQ3+oLlnU2UvtmD/kDJXn+gAVy1F2VcHt3dZIUVMq27cUlUfw1RxnNrW8yVmUY1qI5l7+REol1upvNGPGyLsJGUZktdxld19KUvXNqKp9AxGiBiMEDEY4S1N4IYEN6RANrrrL/C9v+dHZZ84dwJ8Ny+PKru2G7Ooh/dNK/v3eRxrEhdoY1x6FiO4E9C9sjRqPPTvoWu4cUyAtGRwQ4RTpLhdHsDpc+44Ti91YuuUfnUruEc30L5GurMAvlDbI1zxMf2zNErljQ6sU3oGI0QMRogYjPDMA3Vts/H3t4bB91qVYq/5vkLYQykBO49J5J8naSdHchZ9bZO0JqkaO0cKvVrsbcfxJNNKawfXaEs6gWNW5SKNizHjyWJ4gGJ4ahnrWe+i9pRbwWVlRmkNNpjFtM1qmQaUP90M+NYG6BpHDk+CT3oGI0QMRtiDZ07j/Eor2SFOJ+FtLeN1LP3tMLuK59UylB7w8hiK3DJ9Vg+RlmVZobbHq+4ZzdQCbOhjCEvM4eawtgk6N1bAUJT84rK1GUonMYM9e5zuY9vIqWRu0Hfq/OgK+PIvPUa+UzfAJz2DESIGI0QMRthb+pNhc5yI/JxR1k+rRz+Fa1S9eVqsoL9DiL70DFYUX6Nxoumzn6Ivsk3cDE5f9c3jjZh5+wiUpWcwQsRgxH0r8IfNfaFILxuhqEFEs7yxnLJf3YvTx6/OPAvlhxGadKqHhqCsv5+R+jx66tx7ATdHSM9ghIjBCBGDEVsbMxoFcfjcFqpoVKU+wBgnvnngB2Wf9PG9xE+eNzbf3Xpc2bEcbjibP5pV9r0RfHekaZZSN+lbmEbR/tTHWjiGc2s3RVnjnjq2xZ+gbG+xA58CSs9ghIjBiB2f2j4oPp5+StmxQQwvp8YuQPnsW7R/uDSDq2WnpG06KOC9qUfGjY7o+9ZbxCxxmKSftdCN4bWU7dTqlD//YouIwQgRgxH/nzHDeAp4d4o2o71XOga+VBw3IhdXaINAahHvP+1PbqwAZ5pWTduoVk03eLJpNrVCR/JDVgNkExtbRAxG8A5T+tzSSPc6ZbqPylM4XcW3QyzL1aqpZLGeSvbfNDCCBpmDRkjPYISIwQgRgxG8xwydMDr2biEs7wzbbJD0DEaIGIwQMRghYjBCxGCEiMEIEYMRIgYjRAxGiBiMEDEYIWIwQsRghIjBCBGDESIGI0QMRogYjBAxGCFiMOJvktzghNLg89cAAAAASUVORK5CYII=\" id=\"imagef5f5b3b904\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"7.2\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 7.2 78.266038 \n",
       "L 7.2 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 78.266038 78.266038 \n",
       "L 78.266038 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 7.2 78.266038 \n",
       "L 78.266038 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 7.2 7.2 \n",
       "L 78.266038 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_2\">\n",
       "   <g id=\"patch_7\">\n",
       "    <path d=\"M 92.479245 78.266038 \n",
       "L 163.545283 78.266038 \n",
       "L 163.545283 7.2 \n",
       "L 92.479245 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#pa07450cf78)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAAIEElEQVR4nO1cSY8bRRTubrf3scezeJZklsyQSQJhGRIgYRMQdgUhIcSBCyBuXJD4CUhILEIcECfEJkAcQAJFisQSxDqQiBDCkkCGkMkMnsTx7Hhf2t0ckOrV16gDkVje4X2nV/2522V/rvdevaqyOfHWo54hYAHr/+6AgCBiMIKIwQgiBiOIGIwgYjCCiMEIIgYjiBiMIGIwgojBCCIGI4gYjCBiMIKIwQgiBiOIGIwgYjCCiMEIIgYj2H+6YLvK7k5WgRtIFpWdshvAbUoWlD0UWQEuZraUnQlVgAsZtB+i4kWAW3Y6lF1y48CtOkllL7U6gKu1w4Htuo+zzOD9GK5nBr6u2Q5RXxoJ4CpN+hye9gzDMIx6i77yVisEnIwMRhAxGEHEYAR7S/8CXOiPU1z4rYV+eq7YpezCYidwHzXOV7ZZQ41DWtuNou/1wsE+24u3lZ3t/w243gTFnoTdBG4gVoJ2OkrtWhvjUtSieGZbLnB6zOgIYYzstCmeZm18v8novBGEV1evVPZSA2OdjAxGEDEYwbatNlz4eGZC2W4uCVxkjYZtFDNEI1QjO7qKrkfPCmM+LrriKLuVxlRP8wSGXcG+lHvJZS5l8L5ZX8LuaW0TPy5A81h/tB3qq3WW+5wopq+VddSujvseqn0Z1279Gd8v+C0E/zVEDEYQMRjBPrpvE1zonab0bm3CN5XfXFd2ZyeWSsI2OdWVNUzZdLQr6NBHNywre/fgUeBee/1mZadyeN+ZXRRrInmMGb3fY4pa66HfXDODn0mPS3qMMAzDMLXHhMv4zMgapdOR3DJw3afyyvYcBzgrFlP20fsvRc4QsIGIwQj20Mc1uBAu0Ay8ONYPnLmszV6/igHX1GSd+Airtu6RY/SGY6PATT/WrexXpm4GbuTJL5VdePgq7EuY3Ma6KUwf693o0lq30uzd9FVf1wrkUkMd6FKiMXJFqTjOwNvabL1qYJ4/mkope//0ecBNvEB9TSyg65ORwQgiBiOIGIxgz92Ovn/i2dPKXv/ULHBWUlvRctHfNXduUbbZwCqqpfnQ4uQAcO4SpZqjj3+Nvbv8ImV6u1aBOu/pKD1/6hvg6vfuhHZ8D1WYew9iPGv2UUpe7YsCF1ullLmZwnS9kabfcaML0+UDkxllh1YxftmL9P6NCVwhlJHBCCIGI9jt4TpccM4UAl5qGFZ3RtmLNwwBVx6moWr7XFHHfJ+y/ev/2UN0n7d9C3A/P0huo+s933Avktv65Y1JfGgO3camZ+eU3RrJAlfYTm664zS63tg8pfnxCk4BvGJZa+B93tCg1lH8ftvHZ+iZm3uBk5HBCCIGI4gYjGBvGMSKYyhDaWB7DTcBOHM5ZXe9kgMuOzpM9/VlgPMO/qDs+h1XAJe/jvx7cRzTx/ivZA98tgTc8ft6lH3pCK6YLbw2Dm03S/3JX4MrhnrVNr6E5RA3QeUf08bfrTuHnx/g+96CED+DlW8ZGYwgYjCCiMEIZju/ETL/8X0PKjtzAMsDA59SfPHmTgHnVnBDcxDsQZyDtNdTrm2dwGeu7N5MHcVU3ljQQo9dxnlFBkOIURwjPr6AE53sYep3aRTLE8003RcuB2+2sxvIdczQpjb32x8D7/NDRgYjiBiMYDZPj8MY+6JB+uxZ3QYvztcp7T223Adc6Rit2I3twdJB6BCt9JlaBdcwDKO9uBjYOb3a65ZwP6utpdKFm7A0U+tDt1Udo9W1ziO4KpfKUdX21C58f7tE30VqFrmVbXTf+MYzwF3URZXvT1/GVL7vOVq91KcRhiEjgxVEDEYQMRjBfiS/Ay6EtZ3BPxUxDc1EKBbcNvwTcJNbqExdvRNT4g+Wtyp7/zHcHbLhTWp7Fvr66Pu4gqdDL830vIilCe/qSWjnHUpZBz/BlT6zRquS5t3owx2TyiHhKvatf4p+x/PL64HruY7S5cpQcEp86oGt0JaRwQgiBiOY5/Jf6D1pGn5jaRzu29JUYp2u4ua3sDZ9zkZKPo7c4nILK6p7pi9Wdvwwzo4HDpDLtD4//Jd9D0LoAtpr3P9SHripGdqA5pQxJbZXaOXRrqELc236Sjc+jy7UydERs1+ewY0TMjIYQcRgBBGDEc4pZvzth6ILNcZ6qdq7MYXlj7S2e2KxiSt9mTDFhe3Jk8DNNana+8I7twA3uN93JqJJMSv84SHgQlnaLXLyoQngnAR9NSOXYUU5rB3ym54dBE7fuDb0IR4GDDWoL/M34jFoGRmMIGIwwp/+VeefgOdzfCeXaPPAiQXfxi2LXpxK4IavrT1UDY114BmMTu2s8/W3fgtcejc+Z++JC5U9PrMBuOYwHWFOz2LHF3eQSyk1sKpwWR+lrFduRxeq/6vQE85dwCVOawtWRaBkZHCCiMEIIgYj/Csxw5/a+mOIDtfVNrFV8KzIl+UxZX+3sA64gRSVVYaTeHbjnq6voG1pey7evQvPBup/lNbGsGBYXXSOz9+3g4URZd8xfAS4A0Uqo7STvmPQg/T7dxJypo8tRAxG+E9SWx3+o79ng/7aah1nqzN1SpdnFnuA6wnjHq59Odp/VR73/T2O1h2zE4+/6f/64IfutvbmLgSuovXV6sJnupRJG3YIny8jgxFEDEYQMRjhX4kZ/zfePn5JIBfJVgO5c4Eez0rVaODr7LATyPkhI4MRRAxGEDEYQcRgBBGDEUQMRhAxGEHEYAQRgxFEDEYQMRhBxGAEEYMRRAxGEDEYQcRgBBGDEUQMRhAxGEHEYITfAXL7QhwYKxxBAAAAAElFTkSuQmCC\" id=\"image595b562232\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"92.479245\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_8\">\n",
       "    <path d=\"M 92.479245 78.266038 \n",
       "L 92.479245 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_9\">\n",
       "    <path d=\"M 163.545283 78.266038 \n",
       "L 163.545283 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_10\">\n",
       "    <path d=\"M 92.479245 78.266038 \n",
       "L 163.545283 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_11\">\n",
       "    <path d=\"M 92.479245 7.2 \n",
       "L 163.545283 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_3\">\n",
       "   <g id=\"patch_12\">\n",
       "    <path d=\"M 177.758491 78.266038 \n",
       "L 248.824528 78.266038 \n",
       "L 248.824528 7.2 \n",
       "L 177.758491 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#p5c008fc8f5)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAAHfElEQVR4nO1dW29UVRQ++5wz9+n0Qm+0tS1tQcCKiAgBE6ImRiKExBh50IQgMcZ/4JPBmKjExMQYX3gwUfBFE+IDGk3EGDUBRSmUIjcttFCubellptO5nItve53vhKlQqyyT9T2tzXcue/rNupy19xlUxyfv+IaABcx7PQEBQcRgBBGDEUQMRhAxGEHEYAQRgxFEDEYQMRhBxGAEEYMRRAxGEDEYQcRgBBGDEUQMRhAxGMG+m4OtqKvtZLIIXNQmLhUtAdeWntR2zHSAS1hlOs/Ga1qGp+2I6QIXU3Qdz1Bzzjui6FzXx+9fzo1pO+9FgSt59OcpexZwTuA62XIMuPFCStvTBeTKLl2nVMI/v3gGI4gYjGAbCvcjRBMUNhZlZoDLF8mN8wV06enRtLbH8+jSI06ztq383CElCOWpgB3myDbLyPmhW7gJ+oxeJHwTMssZDIUqTaHQsnECdoSONU3kMsmCtlc1XDMq4Vo+A2PxDEYQMRhBxGAEO57CMlQFcsj10Wrg/CnKE34KS1QrSeNo7SxwxQIF6lIJ84kRSFkqlGuUQwHdKmAiiEzTODYRynvTOI5NU3yP38C5mVP5wEVDlb4fuI4bSlq2dfvjDMPwI/R5B7tWAJdrpvOmluN54hmMIGIwgr288Qb8w/BUrbbzUwngVBXVkLEE1pOlWXLNYqh+VBa5Y7o2D1zEohBimei2TemstldkrgP3VOaUtrsjE8AttrDsTpo0HnFywAUDYzZUE387s1zbV4q1wAWf3M9MNQN34Wq9ttN9GPoyQ/R5cx3oC+IZjCBiMIKIwQjq/gNvQKBWffSIPtuC7QEj0B5InMduZGMf5ZDkuZvAOReH//FEw7BbW7RdWtIIXLYzDuOZFvrOldOYl5xual001E0DFwvks8ZkFrj6GLWKxoop4KKBDvN0GedyemSxtp9beRw48QxGEDEYQa3d8R74bV0/lYkTq2rg4MwFenqNXBoDzk9RGayK+FTvT5GLuxNYhsJkIliS+uVShSP/v1A2lbrnPlwDnHgGI4gYjCBiMIJdPYjtCe/kWW1Xn8SDrVpqCbidLcCZ2cB1VGipbVEN3TCVBMppqdN2qQbL5fivg3S/OXLN3yEYp82qKuC8PM3bL+KGCPgc/sK8oe079HjQegj/TuIZjCBiMII9vAXDRs/Vdm07Q5eAC4YKlcPNCu4cZai1iEKR7+JTfaGxlew6XFzyNvZoOzZaAM6+Mk6D0KKQ04iLYqpA3QE1OolcIDSZGdwgYESp++yOjQMFoS+Jf0M3G3hanyO8VX19CsbiGYwgYjCCiMEI6vuLPRDUdh58VduNv+LBpTSVYk4KyzI7T5epGsG8ED94dF6TM3tppc2pxVVHa5byQLEeufAmtsSPp4kLla+qdxndoxpLaytHedAcws1oXiBnmmns2qoMlc9zdazNOHZ0xTMYQcRgBBGDEewaE+v3J9ZT7ftDQw9w7izV1nYCN7FlqqitYMYxLo/0btR2+wf9wHkz+LwC3ClqzYS/NWbPEm1HYvicYd8KXTNOucBsWIT3cALvgJy9Apx7g1YsQ2ueeFwoD5mzhQpHIlR3B553R2cJ/hOIGIygnj/8CpS2p0ebtJ27HGoPBKVL4ya2xU2T2n6t+xvg2m1qo/w82wXcu0c3a3vF7tBGhuHLlWc+B0pPr4VxOU1tlswArlC65weNewX/sdUwFs9gBBGDEUQMRlCdn74FOcP3gqtb2FcIdoPDi3mRwEboTZ0Yh9vilDN6EyPAtQbyyUejm4A78sVDdNyew7eb/21hd7bD2MsEWtznh5ArUBmqYtgOufg67d6wQ+8iNv9C5Wz0yBm8Zh5XTyuhuOVRGItnMIKIwQiqY//buBQVCFN3swavAu9WqNB7FsHQt3HpBeBeavpJ2xti+IrXRYeee7cfexm49jfpydnrxzAxX0zs3ADjtl1/arv/Uhtw1iXquHYdwD26/rHf7+h++WfXw1g8gxFEDEYQMRhBLdj/nxHMGaGfwPBd0hxKZ8MwEhkqLV9Y9htwO2po3GRh2blnjMre/YewJF66bwrG880pkzsoh4yux1ePn1l3QttfHV8FXNdndKz93bGK1y9tltKWLUQMRli4MDXvGVS+/Zou2kS3tQE3/q6OUUe3KvSzOgMlfBX485sUDgb3LgeuZt+Rivd3nnxE20O7MEwNPL5X2zkf77/9zIvaLn6Mc6n+gxa+bq1MAyeewQgiBiOIGIxwVz8YuSCYI0eE0TdIC/b9l7Ed8XA75Yyt9bjJIevhprZt9Se0vXvLfcDFJ9Zp241i2Z1roRVC38NNBu/felDbkdCPYF4Zq9F2sgG/7/lG2uCWb5Zf1WELEYMR/p3SNhyKwptf7/S8+VzDMIxkBkPKtiUD2v5y+AHgigM12jYdvEexnrrGfgp3TkWStA/XsnDehWygW1AKfd+DHe0IlsviGYwgYjCCiMEI974dItAQz2AEEYMRRAxGEDEYQcRgBBGDEUQMRhAxGEHEYAQRgxFEDEYQMRhBxGAEEYMRRAxGEDEYQcRgBBGDEUQMRhAxGOEvke0fzL4fCS4AAAAASUVORK5CYII=\" id=\"image856bcac86f\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"177.758491\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_13\">\n",
       "    <path d=\"M 177.758491 78.266038 \n",
       "L 177.758491 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_14\">\n",
       "    <path d=\"M 248.824528 78.266038 \n",
       "L 248.824528 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_15\">\n",
       "    <path d=\"M 177.758491 78.266038 \n",
       "L 248.824528 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_16\">\n",
       "    <path d=\"M 177.758491 7.2 \n",
       "L 248.824528 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_4\">\n",
       "   <g id=\"patch_17\">\n",
       "    <path d=\"M 263.037736 78.266038 \n",
       "L 334.103774 78.266038 \n",
       "L 334.103774 7.2 \n",
       "L 263.037736 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#p63edc89eaf)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAAJsklEQVR4nO1dW28bxxXeXe7yspR4kUiJulmSJdcXWY7TFKjrIGiDPhQp+gfavhQoijz2H/Q9QNFf0Pa1QNGHAkXQwECApEBSt7BrR07sNIodW9HFoiVSJCXe9taXYs/5Bl1GJDbAVJjv6Qw+7u6Qh3PmzJkzZ/Xl3/w60EbBgKsMV4f24l+7xL1/D7hEaZIa5Qng2kuFUG4tmMIzSHYz+LziZh/aukudbS4lgSu//TiUG989D9zej+g+U6UmcPvbxVBOtBLABaw7+hC/rnH6jyp83VDKkAhKGRLB/OqPEMAWCpyX9UPZyPeAayzboVz6GOcF7+CQGlzWNM32VkLZtUvAJVteKJsnLnBm7QTazSv0zADNu+btV0N57I9V4MZnboZy9RW8zqzRTxdYODEMM09wqJEhEZQyJMJQZoqrLvCRGps+DuUfr9wF7ve7r4dy6YMCXnhYi3yc3ndCOVVHU5T+9x41khZw7hfPoJ3byYVy5hq6r4Ogs+8YnAiuNVlJzcPHjww1MiSCUoZEUMqQCMPNGcxlE923TJLsu51A1xZCJwbqXzepC0Y+B5w7lSfOwUnKmyF3tXFhDLicMGd4TQpl1C9mgCvdZi6qi/OS4VDHjR7220tF+68qHHIGoJQhEUY2UyI6ffLvnnTKwBkOG7e1BnI2rc79Y1w5G588CeX2D64C5yco+urYQjzAEKKo31kP5UwdzZ1omjicHN03SOB13GwFplqBnzkoZUgEpQyJMNycMehGBtnUK/YucO8stalRLgLnbz4N5cDBHbr2Gy+F8uFlnAeye2SYp25jSMXzPWgnjsnVNoq40zcIAfurGn2cl/hOoxfTr6hGhkRQypAIsZkpg5mp9fSXwJWLrVDWhY0fHv5NXL4A1HGFTFOqjpdlaswUHQikAP0ZM5sLF5GzyGyJZjJ9SKawX0Az5QubVHFAjQyJoJQhEZQyJMJwcwY3m8KS/6STCuX73UXgDhoUVc0dodvLwxHeo03gZpo0v7hzk8D1iynttPA7lETXPIdfObtKfRWfz11bX0g6GBQaGhVqZEgEpQyJEJtr22uR2ej5uEO/PkumqVPG/Cf/y+3Ie3oVWq0bfVxV2xvsOmtwRkDQoxV4dh/vo72ITog4Psf6ksWordkk3zZIxGOz1MiQCEoZEkEpQyLENmdoPvm980nMmZ3NkFu6edjWoqCn0F3dv0EJCaWNDnDG/eehHNx8CTh9eyfyGeO3HkLba7UiPqlpPvt19AzuCAbH8f+P1ciQCEoZEiE2M6X3Sa8tD3OTzmdehPLnc9/ECx9TjlPv9WtAHa2RaRjbxU2h1PfoPs1FNG/FD6P76Q8wSyKy22R6G3nhpxLPRMQANTIkglKGRFDKkAjx7fR1yYgmdAwdXE9vhfI7uVfxQpY8YH+6D1T2+nwop+pdLQq+kGPgv/YytM06ucVOyQYu8d6/Iu9rsCPLRlvIEWbHq8UktlGhRoZEUMqQCLHl2vIVuKV7kR/Te8jxW7pPt4CbukM5u8ldzNF1pun4gJdEP/NwLY33uUsucnsabVpxbpaev4MbX/y+phA48GM6OsahRoZEUMqQCEoZEiG+qC3DvpOHds2lhAS91xc/HonUCzLUeguT35IuzT36VZwjAkNIODPpP9ctClyJ9VWcM1hUJ9ERrkvGn5GgRoZEUMqQCEoZEmHkOSMQQsi6QTa0LcQnDLaacCo4nyS2KPzNszg0TdMCZuvdvedaFAqbmODWruAioF+g/nQq2HEvS5y1uABcr8DCIQ5QWmCoOeNMQylDInwtrq2ICZMq7hytohta/ozMVjCDCW67rxFXwUI9gOR7H2H7+iVo610Kh0wlsAqDfvtjaiyhmfLTZIo8G80SHKeOCWpkSASlDImglCERhpozBpZhYFzDxeyQ+SQlF/cmBNeSFWzU57HMhcOK5Rjj48DxLA+x5IS5h8nMQYGuzexhWCVgO43+LrrPRm+G+pk5XRUdTVPlKs4ElDIkQnwrcCY/OJoF7lKGiju2LmPUdobJwd1PgJuzKbHAW8dCj9YzSowTd+gCG93nXoXsndUQCpPx68TiX6yrhisU/0qrFfiZhlKGRFDKkAijh0PEaAAzofstdEMnFygcsrCAZzcGoTdJ0dfnN7A+RPEhHRnOP57C69L42doligwnuhhRnrxDsnEej0xzF1UXwh9qzjjjUMqQCLFFbbmr22zhCnzWpKo3v1x+F7jfTd8IZb4a1zRNs5/Rajm9gptSqQa5oaKb3Z3Ar9Vj9cbMdnS0tbmOUWOXRWrNk+jrRl1xi1AjQyIoZUgEpQyJEN+ZPmY3/Qa6j08dssVpHXf2xXkC7vmIXttWnlgDzvrbg1A28uhK55oY/c1uszN/Azbo+mPif5O+lHDkBDO2Y9r0UyNDIihlSITRzZTgzvFqAoGJY5ofRf7cqZz+EVdXQ9kZw1V1ukKrbq+Ebm9wD6O/1hKVx/FzeIyM99TDYC/k14qvaODJCvwInaZpI5stNTIkglKGRFDKkAhDzRneOG3eFyv4FmCTvV8ibeKO2c/ztNF/q41R21s//FkoW010e5M7FEYJFrLAtddoj7A1j18js/xtaNcu0XwzfQd3Gq0NkrtCsoSTp+87eR4LGTdPaIJx+zif+T1qJxqn/4nVyJAIShkSYbi8KYd012igi5h+SO6rg6eEtbdmqcb5kw5GRo9nqQtTnx0Ax99EmRVzmlZoI0h3cQWebKApql8gPlkXjh0weebvWIWhM0WRhGoXjx1wn3hyHft9xKLWvjJT/59QypAIShkSYag5w6qR7sbvYexg8rfR5c/+XP9+KJ/MoP7Hm2R8j9cw2jruk0UPUng0rPYyvcFy4p8Y+fU2n0C7cI5cXWMLK/fw4hnWXayFnrTJ9hf+gdXeaq/OhXJ1Dt+gqe+fLkosQo0MiaCUIRGGMlPOIrmFh2W81GpTYkHhLxg1zf3hNsnCPXktW/9bl4Hzx8gUNi9iZLbP3i4pmiUR44/p+IBfOwLOuEZHznRPeBOl8LZNjtY5+h/rVTRhqUPieiVxVyoaamRIBKUMiaCUIRGGmjNurpJt/tXc28C9ufKTUHZ3VoEz3r8XeU9eFUH/4D5w3NqObQClFVg1g+j3Fv/3PvcfRnLBxqdfcfX/hr1HxSyTDfRf26ffzASokSERlDIkwlBm6kGVNnQ+nFgG7q3VP4XyT9/8BXAL1iuhbL0r1JEN2Mb+9StA+R89CuXElW8At/UGRVHt6jxwZhdd1ESfDF7qACO6gUX/x9QXQtSYFbA00hhxsKtkHJtL+DO69mjJt2pkSASlDImglCER/gNl88iG0uP6kgAAAABJRU5ErkJggg==\" id=\"imagee5ee4d0d05\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"263.037736\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_18\">\n",
       "    <path d=\"M 263.037736 78.266038 \n",
       "L 263.037736 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_19\">\n",
       "    <path d=\"M 334.103774 78.266038 \n",
       "L 334.103774 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_20\">\n",
       "    <path d=\"M 263.037736 78.266038 \n",
       "L 334.103774 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_21\">\n",
       "    <path d=\"M 263.037736 7.2 \n",
       "L 334.103774 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_5\">\n",
       "   <g id=\"patch_22\">\n",
       "    <path d=\"M 348.316981 78.266038 \n",
       "L 419.383019 78.266038 \n",
       "L 419.383019 7.2 \n",
       "L 348.316981 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#pf07da0e887)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAAKFUlEQVR4nO1dzY8cRxWv6p7P/Z6xvbve9bJrxx8hNnIcmw/bBHEIQskBCSGFMxAOKBfEhSNwIZdI5AD/AVKQSJAQCIJAihPHDmABdmKCSPDXrte7a+96Z3Y9HzvT3cWt3/u9TVdmjIhqpfqdXulVV/X0m3rv9atXr/XcSy8alYVsjlKB6BpS5wOvdoGXq2+l9OrJEeS16brmOA66eYTG0cUEeOXhdkqXCjjfgyu7oD1xKU7pcAt/VL7eSenakQEcZ1qndHsiBt4zpy+n9MWlOeDVr1WooRXC8kyDbJbHxw0vDIfgheEQclau1HcMJkDlV1oOUzo89xfsy+hd/8gec1S099ruLWDzHZwD1oS+B+1kqEh97z8AXnTjVkpX3sYpmOZX5sxx4F2coznXV9AOBjZba4FfGQ7BC8Mh2NWUBbsfuQ/tM2dvpPTF26ex72//ndJ6eAh40fwiNRJ0H20IR0k1RLtwzKQYQrs7TD9zaHWj5zk4gi661rVbYyldWsX5OqOsr1RZFtXvV4ZD8MJwCF4YDqE/m8H035OT14D15dF3U/q1A58BXuX+ekoHrTbwwgOfoOEHisDTi3epEaM9MdMTKZ2/uYL3Wchjs0Tjxst3gZeb2Udjbm4CL67V6V62MORSXkI7kQmLjZDwK8MheGE4hP7UFFtyV2tTwPrS6D9TujO7BbxggKKhiVRTmg0a4H8j2aC35XB6EnjcY0zq6K5K91kNlokXCvWSp0fA1ZLE5iGMDyT5jI7/A/zKcAheGA7BC8Mh9Be1ZYp6vV1WWZiaXIe2Zq6lajaRFzGXtRMBL4mYOynsiWEhj6BcwjFF2xi6cZ3Dn2xyvbmojb3YLymwh2GL0vpwyM6EF4ZDsKspucS4mqoPAiuvScV8YeI/wLsysj+lgwaqKVMqsIaYkLcTjJrqLlNvibhOC13A+eLtXPWoprbEzldcZKov7uM12wK/MhyCF4ZD8MJwCA+dkBA1UPfWYrIh00V0ba8oshng5iqlTIFuQbcxMor3gjcDNkNim30he2akXepY5uRDFvE67toGvQ3xkfArwyF4YTiEh05I0C2U450uZRm1Te9vuaCaRD/5tgw8ttljxMaTfFu3Qbe2PrqTUkonqCbDNs2R5KRr3fP0AL8yHIIXhkPwwnAIVpthhC40gyyqKnRozBTlE+WbwPv18adSevCV68ALmV0wU3uApwsUKjFldIkVj/aKEIcJ8T8GdiHI3umzYauC7nKhRnNE48JmFaivbgo7GGUbFL8yHIIXhkOwqylxdGus2kjpjQ3cXFpoV1P6u5WbwHvuGVrGh1/BOeLVtZQOpseRyZIHjHBXdWjZMMJRlIlIvWqhwrZFihm2nv40dcvjsxi8Q+omOIqR6AO76Te9Ny8ONqwVVBb8ynAIXhgOwQvDIdj9ui66YfUF2u6Sx4v/eOJzKX3u+cvA+/ap8yl9fgCPBScsQUFHqJchUtvFZAXIvbUkpiklbIoIuUTzt1UWusPU97EXsF+0QO33T2Bu8Z59dDStWMbn1FHeZuwIeGE4BLuaCsUbOGe9/nfg7X2d6G889k3gvfDkqyl9oXoQeEmb3o51DVPyDX+zlhtGPMlAJiDEQt2xa00Bc6psrm1zD/1Xhxay1dnEefxPX6zQZlqngWrJ9u/3K8MheGE4BC8Mh2C1GdUpPK/w9My/UvqvZ08CT1+4nNKP/Bx1duMsRVw7cxiZDW7T0eNo8Q7e3OxMSpttiWlsDrnTZ0uG62MXcOJr5KKan2X3q1zF53TvFL0CbKuW4HNtdwa8MByCF4ZDsNqMJ8YXof3V0b+l9B+OfB541QtEh+fwHWSlSzp09XGsdjb+Vvb8hie8yQwTvtMnEuO29WX2RluS1uZ/eAbaz+56I6X/rLIP8QV3MWkv1xhL6UQ8YZkMB+Nkcjw+dnhhOASrmoqMKOCYUCihfgj7VlU2RnMUmX1wBnfFxn+afZ1u0jFlM1CydBT+oozwsmNl8eJS5jBjn8VKC79bOJrSVfV+5nXJBP56zbzusCP6Co3K4VeGQ/DCcAheGA7BajPWtvDcXtuQe1f6ZK3nSV6epyyL5z51AXhvzj6a0tGtBeCZVosaMomNh0OkzRAxh7hCv8Ncz050frSCFXfeOncspW02sTE7lMnbft7Pu7Y7Al4YDsGuplr4tlzS9PZ6chJ3vjDeiqj/iSriPPX8L4D3+2NfpPFFYS6l2X9F7PTZIrFa7vRtU2Mfjmq+ge33erpMNcfxjd+EnO69yK1fGQ7BC8MheGE4BKvN2GhhCCJg7/mHBlC/31HZVXZ2XyVbsxChk9iq0i0MzM0AT61RNFTLEAeUqIgtPKU0q9YjNXhYobOI1x5gJsfIjbbqBZ0RtEn8jJ/p4+/uV4ZD8MJwCFY11dhANdVlPtuxMr4tv6GPUEMkBJTOk4/4vbefBd5EzD7zc7ACvME6JbUZmT/L3Vcxnyli32CT3uSF06vMLJ2feOfKMPAOJr0dS47wDUAl+Yf7ZoNfGQ7BC8MheGE4BHvi8wPchF+LKTo5EqDbp09RhNNcehd4SYPCDDO/FOfvQtLiYUtodHbuAgpLSojwx7Y4aSvbRdVLqyk99SbajLBO10lbA+Pb/tK+FvrOhBeGQ7CqKd3BNbbSHUvpZtgCXnsPucGWPXc1MI9nMFr7SPUV7mLUFCoftLeyefI4sVRpsqAkQ7yfIsoDSziHXlmT3T8UsvhXrkHPLRryUdsdCS8Mh+CF4RD6qsTWTCiqWc3hlyAbEzSUzWbw0IRSShXvsao64pNucZk+/xY0UJ+bPCtlkcP/VLAp3G4WSjGHHwdeVKZxCmt4XXwPv4SZhUAElAub3mbseHhhOIS+1FSbfdtmT4if1lk/SsvRlmNkRC30kLulsnYtz7UV1/GqB0ERN4VMF31Nrih0V3wUN88KP5ZEZFhlg394Vws1Zfp6quxeHu4yj/8HvDAcgheGQ+hLu91sUUWc04MfAC8/S6GMbZXRWCW0eEV8QZIXjGyL6GqJ7TRO7gaW3iQbEi/juQpeaFIpLEIcimPKAS9SKatcqGzUDtP2Xq6J7mtzSvbuDX5lOAQvDIfQl5q6vDKd0l/fjXLcNUxqKhhF9zFeww+5c8iqCBzhQapOkwyJHK7VGusoasfuwyKN+i5tIPEClUoppWS7RxQ3eEIEPosoO4XMCr8yHIIXhkPwwnAIfdmM2jJt2C8fwW+l/fjwr1L6W9//DvAO/+RGSkdLyz3PF1+fT+lwCI+0xRsbsjthHXm2rxlzhGP4m2zXlZfIDa/P4b3xJIt+4FeGQ/DCcAh67qUXHyox9MQp/Erlj2Z+k9KLEbq2P/jgKym9fgnrnR94mdxOcxtVWHKUXNuN/ZjQGpUppmpE3Za4iPHWkZsUARhYwISI9WPi05QM1ddYVYRJLFpWO0Z5wavHxenass+13fHwwnAIXhgO4b/ws+0uRCYrpAAAAABJRU5ErkJggg==\" id=\"imagea32b157982\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"348.316981\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_23\">\n",
       "    <path d=\"M 348.316981 78.266038 \n",
       "L 348.316981 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_24\">\n",
       "    <path d=\"M 419.383019 78.266038 \n",
       "L 419.383019 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_25\">\n",
       "    <path d=\"M 348.316981 78.266038 \n",
       "L 419.383019 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_26\">\n",
       "    <path d=\"M 348.316981 7.2 \n",
       "L 419.383019 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_6\">\n",
       "   <g id=\"patch_27\">\n",
       "    <path d=\"M 433.596226 78.266038 \n",
       "L 504.662264 78.266038 \n",
       "L 504.662264 7.2 \n",
       "L 433.596226 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#p5b63fbe3ef)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAAHZUlEQVR4nO2dW4wURRSGu2d67jN7YXbZ1UV3We4i4aJcjBglPvigiUETDDE+GRPf5FV8lGgi4ZVoNEaDTwQJXtAHghoTggjIxSy64soiy3LZ+7Izu3Ppad/q1F/LtD0bGA/mfE+n+tRU18yZOnX6dHW13b17j2cJLAj91x0QCDEGI8QYjBBjMEKMwQgxBiPEGIwQYzBCjMEIMQYjnFoqeyHKnFQihi5SoUIEMyyhRFnJ8UQRdVqbUacMuniEyvlCFHS3cnEluzPG13BtKNrT9J8Lz+D/r5x2SddYwmaKVDekfz/LshJJ+h7FAp7fdelzlWIY+5LXykYiSkYGI8QYjHDKLTg0m7JTJCenQdeRmlByV3IEdBWPXMNkOQG6nEsuJldGdzNWSCo5FkY3lXTIFUQbXNC1LKB+bkr3gW5VbBDKV8pNSu4vtoAuX4kpebiUBt3hyyvpfOkc6DZm+5UcstHfJEPU75KHburIteVKNtPlMjIYIcZghBiDEfbqr94C1zV2tVHJiQEjZEtQVc8wo5OnOSMyhbr4KH0uMYTzQmxkhjpTQS9qF7W6NoarpXk0L40uj4FushvP72ZovrHjOPd4FWo30YfzWdtpmk8N1499SaGymKY2C83Yb/13m16Dc7KMDEaIMRjhTPbOgwMr3u5Rsjs5WdfOmKGe30oJ3TG0fo+61rl2wHCFlhdsrUbs36vclvLTj0BZRgYjxBiMEGMwwomPoJ/U54lw23zQ5TZ0VW3IjVI7RnYAwjkbI0srktOypnkMe3UfHs5jtjd8fYzaL+Pn7IiRUoaOYgdmHuogVRz/m5EJCm0jN2/hOaYLSq4MDeM5wjSjVXKYRtFxjp6GsowMRogxGOE8/sIZOHDxFIVboes4xJKXyYXZBcM1TOW1ghEi6hguxcvTVajpbvThbkcN1xOlq2U7lQSVFzPqalf2XgSzCtEb9B3tImawIbR18CrbbaFMRSiBwa2X0K7kz/1mBUVGBiPEGIwQYzDCSYcLcCB+tl/J7vCI9b8jZKRfK+7t65kfSyaNsnY300ib2NlmJYezmG5yR0artxmoJ0JdEGMwwrky3YxHmilks+rgpuwYhYVeyQhtA7qQmjDatCMUhnqlolmbPpbP4wGzrKO5Ij9CDRksB/qUUBfEGIwQYzDCdq8thrhs/S/blJza2wiVY9+crE+v6oke6vrNUTXcBdTDWddn/gjF41iufnah3ogxGGEf7++E8aavPf345mao/Ptom5KHRzAsi16iIbfw4BjoKjVkLgOju42ACwfqRVA3ZV6dy8hghBiDEWIMRtjrDu8Eh9sQp7WvNyZxXsiNUaYylsFs76MLrig5EsIQ8Vj/QiXPP4DPbqQOnKi1z7OZYya2JszQ1tb+x8b5wk10SeCOT1hV2bAKijIyGCHGYIQYgxHO+IUsHBj3CdkdzW2Wougn82VakdFo6LYtpxUoU2/iSoqvt65V8oLPcOVG7NuA6RdzjqjDHGJrK1fmepUz1ZWCsowMRogxGDF7hwSf9Wc6oUHMOP56bbGSy014x27l0gElL8vcAN0ba75T8qG21aAb6nxMya3vHw/WsbuFkXLxuyvozRSq6nSKKfyxZWQwQozBCDEGI3x31fHMDIBfDKevER7DZnt/7lJyT7YDdM+vOavkdxd9Drp3tj+n5KHRTaBL7/+pel/uRjqkBirmAupq9fBJZxkZnBBjMMLXTfm6pVmVA55wBJ+d+OLkOiV3bMY7hO91HVTySy+/CrrGnmVKdnt6g/bSF9vBn8PTnuuwQ8ZOB9qzJObCgsrMjFWN8IolSi5JaMsXMQYjxBiMqGnDSF/0+SXg/GFZluVMUPbzg/NPgG79xktK/nT1J6Db+toOJS/eYQVGX+hsWZjWmPVMoV6vUlXlO0eYuBlto0tjnwsZGYwQYzBi7m6qBlcUtB17ABcr7H1wi5J3PfAl6LY/dUzJR17BxXZN+6pneP2yrfUgdP6ikpMr16Ku3p0RqiPGYIQYgxF3LrS9C5w6Q3cPf8guAd3OFtqRZv+z6Hub9t2Z81eepHanWzAkzvyp7VJnLHCrnL0QqP34OMbLMjIYIcZgBGs3FdZetfBhH4avi1bQwoal7UOgc2vIouqYWdvhhynULjZg3cEtdMAz9sptP0o3wub9+Dfoit30jEt+Po4FGRmMEGMwQozBCNZzhs7oH/j8W98i8r27Og+B7sWPXoeyfZ3So24Kw8nsacoat5zDTdwzA5TFzbXj+t1ClsJZr4w/Y76d/uPlZzpBV8xo+6TjV5KRwQkxBiPuGTcVKuNV7okJei/Dmvhl0HW0jkP55l/tSo4aa7rSg7TGabrNeD1RG7mmHC73smztVQ+2kQjO3+dpMupgza6R+ZaRwQgxBiPEGIywu3fv4bXXQ0C8DkpxOBFj4+JefMWbvt+78dY6q9SgbUDs8yqfO4bPry0jgxFiDEbcM6GtiX1VW39kKo31SPn7GXlin4UcMjIYIcZghBiDEWIMRogxGCHGYIQYgxFiDEaIMRghxmDEP4yv9W6XmSroAAAAAElFTkSuQmCC\" id=\"image5ac9817ac6\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"433.596226\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_28\">\n",
       "    <path d=\"M 433.596226 78.266038 \n",
       "L 433.596226 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_29\">\n",
       "    <path d=\"M 504.662264 78.266038 \n",
       "L 504.662264 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_30\">\n",
       "    <path d=\"M 433.596226 78.266038 \n",
       "L 504.662264 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_31\">\n",
       "    <path d=\"M 433.596226 7.2 \n",
       "L 504.662264 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_7\">\n",
       "   <g id=\"patch_32\">\n",
       "    <path d=\"M 518.875472 78.266038 \n",
       "L 589.941509 78.266038 \n",
       "L 589.941509 7.2 \n",
       "L 518.875472 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#pbad3f2c649)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAAF30lEQVR4nO2dy28bVRTG52U7tkOd2ombkLTNizQNoRBEIlCo2EQsyoJWiAVSuwA2YQUr+geABAKFSkiAhIQoL4llJRBigbqCioSQiEJUkpYm5B3ych6OHdszw+6e+w2yFRa0B+n8Vuf6m5fzzT1z5sxYMdu+fM03BBZYd/sABELMYISYwQgxgxFiBiPEDEaIGYwQMxghZjBCzGCEmMEIMYMRYgYjxAxGiBmMEDMYIWYwQsxghJjBCDGDEWIGI8QMRogZjBAzGCFmMELMYISYwQjndMsf8EFNaE/FjZEMaI/HJ1Vcb++DFtLipB0pu8M9rwjjw3ZMxa7vgbbp5Wj7Jp43G66r4lUP95f3QzD+NX+UtlmKg/ZcYrTssU4U0ip+OPIXaFWmqeJbxSrQpgpHVDyWbUZtm7aZK+FxysxghJjBCDGDEebA1Vfg9xnTI5RffRMXLtVq+b6IPoa2bBW7kcBPPrTtOFncqE+rGcVUCTQnTvtzN/C6EG/cUfETTXjdy7phGHfGl1V8omoJtNbQmopfvf0MaDNrSRUPtEyB1h5bUXFXZAG0Gpuuu1WmC9piKaHidbcaNJkZjBAzGGH2n3sLckr0yoiKrVgMF25uong7C1ppHqfqncROJWHsrm+UXdZpvBfGpYXF/+SYFCam5f0zj6h49gzOBZkZjBAzGCFmMMJ88KUhuGYkf6c2x24Tloi5FHlnYRVqpN+7puLpNx9D8TiVeu5KFNc7sari5XnM/SffydB6N26CNvUR5V4zhG2UjkvYqvHHJ4yD4BxtgrFfRd/fvXn7QNv4N9gdbTCWmcEIMYMR5tRsPaSpFyfPq3huoh4WtnNUpsXnsWRLv09p6tZnPaD1NM+p2Avc1s9kKDXZVzBN1Q1Tibp7XwK08298reKh6wOgRb/HO1tPa47m67A7UNdDd9Jnm34BbXzrmIpHZ4+B5nl0Hrt7Dmj2NrUVoit4vqcmqKtQjNugycxghJjBCDGDEc7nmT74YG2XnoR51dhx7O2j7ujIaAdoaS1O/IBPvsY2qYQ71/8TaL8tUHsi/vQmaIvxlIqTN7BcfffyWRW3DI2AZna1w3j5NF2Lgp3oxZlaFX94/UnQSoepfjfD+LdwtLFVXQAtlKT1dhNYyocz2gUscCwyMxghZjBCzGCE2fftRSi8d/P0RM0ysSYf6/1Cxe1fDYLWMYh5uxzWqU4Yzz6VLLOkYRz9jp7mmTl8q8QPUY2+0473FYHDNraP07L5FIqHtC6HHzg1M53asmm8ZtmOds2wcJuhEGn7eXwDpFTAewsdmRmMEDMY4axuHMJPtDnuOFjOrWsvlX0w8AloL7/+gorbLk2Cpj95yzdgSqkfpunvXP0ZNH3yWzXYDpkbvF/FsSVME/fMYamZ2qHv4VtYT2brqZWx8QBux6qlYzMDuc/X2iGej9q+q53jgfWcQIkM+yurCHccMYMRYgYjHMvyyopuCcuwjzMPqfhiCp+8fXPhbRU/33sBtK0cvQgcDWdAS0TytFzDo6DZWjW71YrnTWSTcnEujdcB08MnlJZLy3oOLrvZTZpVlwfNtulv43uB3oWGH+ixBK8vB0VmBiPEDEY4/7hd1aZccLp9OkUd3qVmLDW7YvQy2Kkkvhg2vt6o4ob4NmhLWSqtV/oDKTNMYzuKJeGz3cMqvjyGL0CYLr6XG9Z2mTkZKFGPUPlqB8vXA2abimkp2CaugMwMRogZjBAzGGFW+v8ZplU+h+rtAMMwjHCE6lDHxtyf0zqX4UA7wNZK62CXWM/F3XX4u4pr06203p/4NC3YfS2l6NjsGL59F9zn3URmBiPEDEY4lcRKd53Bcq5YoE0VgwtrFPYr7rIsP2ZbYAzppXkPtOBRh4z/BzIzGCFmMELMYISYwQgxgxFiBiPEDEaIGYwQMxghZjBCzGCEmMEIMYMRYgYjxAxGiBmMEDMYIWYwQsxghJjBCDGDEWIGI8QMRogZjBAzGCFmMELMYISYwQgxgxFiBiPEDEb8DWduf312vqwsAAAAAElFTkSuQmCC\" id=\"image441ee1ebaa\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"518.875472\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_33\">\n",
       "    <path d=\"M 518.875472 78.266038 \n",
       "L 518.875472 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_34\">\n",
       "    <path d=\"M 589.941509 78.266038 \n",
       "L 589.941509 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_35\">\n",
       "    <path d=\"M 518.875472 78.266038 \n",
       "L 589.941509 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_36\">\n",
       "    <path d=\"M 518.875472 7.2 \n",
       "L 589.941509 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_8\">\n",
       "   <g id=\"patch_37\">\n",
       "    <path d=\"M 604.154717 78.266038 \n",
       "L 675.220755 78.266038 \n",
       "L 675.220755 7.2 \n",
       "L 604.154717 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#p31735946a1)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAAH8klEQVR4nO1dS28cRRDumVnvrneX+J14BQnEUaI8kHiIA+IQ5YAQV5C4cUHKiSO/Ag4c4cCBCyf4CUjhlAMSAgUpEYKAhePEWSe2Yzve98wOt6n6ajMd72JbRajvVL01092zNV1dXV1dEyx99nnq8hBgMewTXVmTTCLjcm6NLhCtJUVGi/viCl08KA+AV9ijBqt3sS8Te9jIYILVOYnXBqzaMBb3RXSt7Bv/b/gzOOdcu57QZQm2F3bF/8Z5uRzDkcOEoQgmDEUo+JiDEupQrjfjar7uSyNfpViMekTzOck55yaa1EZnFt8bPp/snRRtiFcsrjEdPsB+F7fo4ok95KWsmIo6C+0n0845F/by/xsfbGQogglDEZ6iplCn8CEeCHUjVQxH6hm1YUx0+RFWWmlQpTtLaD+26vnv0ZC626HHlCqFq59U/BsB61vURV5pm9SkfL7OAv0w8P7DCBsZimDCUAQThiKMoNGcc8zS5S4G53AO4brWOedCxhsIs7d3jFUf4btRfEwXJ2XhVmDzQm0V55rJDexAytwa/Rp24NFZalO6aiY6vBLkcddJGmLfog4ri+WBbw6xkaEIJgxF8KspYbKlFVrJdual2mBmr1BTfPhLM7A/Syqmv4vvRmmbVEo8ifd1p6nSqI339auoQ8EjINRNj7UvPappwExUqW6K1KY0pX2mvA82MhTBhKEIJgxF8M8Zcg+wz92YckJhtBBxmr+X6NKQ7ebJ3rD7wh6ykhrp+uYpsdP3GDtQvUcVTTRFZ7gXVzwSnyekKc93M0PBG0wws9fcIf9NmDAUwTuIwk6+rKT3k3t0pQbjq3O5SYMNogrh91UeIq83w8zeigwkwHJph6upBHhRl/6CMM7fXJJeWwhIEGoqZeUhM9ejsm1kKIIJQxFMGIrgnTPkbp5vmQ886f4MgnxekRoZVNCP0q+VMrraQF1f2qT3qLOIPNlGMGCeWRGoxoPMBgXk8fktkq4ST9CFZ1rwwkaGIpgwFMGrpnxm6JD5us9Vtoxb4mP6jTMrwPqpfSaje8vY1fYrZFt/8uoPwNuKq1D+9s6VjK48wIfqniDVGPQ8K/kRTFR+6Sgqy0aGIpgwFMGEoQijBSQwDFmvvJzmuxUCYYVG29SFm4068GZukP24eK0BvPWdExn9ZfEy8DqbuC24uEzmc3sB37/KQpN4jRrw+M4f98Q651wUj7md54GNDEUwYSjC2GrKh6GVuiciga+A2zt4Vuv09/czOl7+G3jzt5ep8JU0l/MNyvTDN6G8u0aqqbqKy2oZBHHYsJGhCCYMRTBhKMJoc4YnHmHIXcDg8/YmddpCu3LuNvCWL17I6LKYM8ZFUsTOzP7KjheX8NreFHPjHIIpK2EjQxFMGIow9pGAkRDk0M65qalWRr839zPwPq1dyuhygDdGxxcyOll/4G2+8NKpjK7dw8DYQodcAivvijQIrEnfMbmDgo0MRTBhKIIJQxEOxR3igzRzQ3bGrBFPA6/Q5tFvIlvDU+YJjr2XFzO6dhO9v413ns/o/owIenhA7pGhnc19t75/2MhQBBOGIpgwFGG08xnjKkpWjwyMqxbJgO+LyLBejd6Vf+PNDvv5c88uBaC4wja2z7PjJGV5GDD/XMe4sJGhCCYMRRjp6PG48GXjeX1+NaM/OvYX8L44Te/KlKgzmpvN6GRzC9sr4GN15qi88QFmCguY2io/xAfm2RvGdgWNABsZimDCUAQThiKM7w7xRYB4kApX+E6fjNZKiNnWSmwqKNQXgRffR7cGtBHjOY/2HL1z7UWRgadBPBnozQPXQpGC4zD8ITYyFMGEoQhHsgKH/LDimO6NdfKafhxhgFn9GvPMhuO/NzxLwcwtfIjuDNH9Y8DCuOBxU+WMABsZimDCUAQThiKMb9r6zvANpatgPwh3yHSFzuY9V+gAL+iTPZl2Zb6IfASvXYJyZZ0aLXSw43svkqdWprnYp7V+YLCRoQgmDEUYO4htaAjDWTGZheDJtHPOXT11PaPX+tNYZZMC3JKNzX13s1OvQLm0TY1unUfbOmEJvnyJH4MD22nLh40MRTBhKIIJQxEO7HyGzwyEI30iG001JJN1pT0PvIHYwctDdAz9GM1FfCyeJLIntgxlQmTgmWn7/4UJQxGOJCCBQ+YUX+4ez+iuSGwbzU9ndNxYz62zefk8lNsLQW45DQ9I9/BqLG7q2YMJQxFMGIpwKOczhl0ljBSm7QtFcnMkQvmulS7uq724gu9Uu46u4cEU+TlKq+ITxbybHt0/9EwWkPBsw4ShCEeTVYdBem0HLFipLJbD8cqq2w9q3/0I5eKFt6AcnqRNq8ShmoIc7tERL7kFbGQogglDEUwYinAkR495DKvMTnN99xx1JhTftjj3YkYnf+DZDR/kJ4FaDynpcHEEbzOHN4vQAU01NjIUwYShCAeW19bL82SnudOiYNf65A7wWmfpqFhpBDXVWsJGghYt+8ME9c3Y5qwvbmy8Gm1kaIIJQxFMGIow0md+IIhN6F7fWQZusco6p4sUqLY0uQG8u7dod88TNzCE2m0MVONeFvFpDeirsKy9gOf1fApv6GuenjptZCiCCUMR/J/58aw6ExFYwE+DShUG37IVquBeczqj35//BXhfX307o898g97W5Pc/n9xp59zsb6jUWvNk2nZn8VpQKeIkrvdTPjw3jFRT7BmHqvC8/jYyFMGEoQgmDEX4B1/kPsomx3LXAAAAAElFTkSuQmCC\" id=\"image3b25f15a64\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"604.154717\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_38\">\n",
       "    <path d=\"M 604.154717 78.266038 \n",
       "L 604.154717 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_39\">\n",
       "    <path d=\"M 675.220755 78.266038 \n",
       "L 675.220755 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_40\">\n",
       "    <path d=\"M 604.154717 78.266038 \n",
       "L 675.220755 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_41\">\n",
       "    <path d=\"M 604.154717 7.2 \n",
       "L 675.220755 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_9\">\n",
       "   <g id=\"patch_42\">\n",
       "    <path d=\"M 689.433962 78.266038 \n",
       "L 760.5 78.266038 \n",
       "L 760.5 7.2 \n",
       "L 689.433962 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#pb3c07608f2)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAAJ6klEQVR4nO1dS28cWRWururqR3W7n7bbdtuOH4knnjgJzjCZQSRiZhN2M0IIjTRL2CBWiFkgsWDBCrHgJyAQCyQWoAEhEBqG5yhR5pFkEjsvJ7aTzsOPdrfbdj/qyQbdc787aoNKBbpY91ud0ldVt+1T95xT5557Kjb5kx8GWgTQEx6TfVcHbvoXMSbXX0wA1zpjMzlbbAOXMl0673oZuOwjknfOecDFmwYcpzdp/IPFDnC+Tecmnpo4/g5dd/Htj/F3u0km/315TosC+r8/ReF/BaUMiaCUIRHiUd3ITJJ9t3cyyP3xCpOrV3LAFS+8wOTNxQJw2xN0z4FGDLiBJw6TD6po67M1dIPlm/tMbgi/zefcy/CVHeACg8Zce7ME3HqjqEUNNTMkglKGRIjMTF2aucPkRxWc0u0vnGWyd/kGcOn3bzJ5cusEcI35LJPLv7sDnFcnkzKxvwiceWu977kl5xRwzZMDdN7SXeCMQp7Jd59OC/ek0FazMLQOCzUzJIJShkRQypAIkfmMC7l7TM4VusC989o3mDx+Ga8LXApf3QyGqK0pCi1zC8eAM2+RXD+eAi6bmYVj6x7Z/t3ZAeC8JIbMAIPi3tijNI5PkbVmK59x9KCUIREiM1OrvWEmf6eEYWjwUqvvdcb4GJPr00ngnLxP5+3bwPHhauIA37jjbcFsNGl8NzUClLXV38Q0vkzZ2ECwZocYt9BQM0MiKGVIBKUMiRCZz7jWmmCyWb4P3BcnVpn8SEMEHQqDdcF86z2yzDHXBy6IERcIj5R4bixJvsjOobUvf9hgsug9mnP8jdEvOblIFkgBamZIBKUMiRCZmVreqjC5NrEPXC7eEU9nCPboXN3Fqe9ZJLs5fMuOW0SKYacumCktzq0gCdbFu40mlYeTpfvoNg4SRPTWzUPNDImglCERlDIkQmifYRXQD1QGyPYPGZjW+Hr5AyZ/8yvfBm7gz5Q68Q20y36aK3LIY0bX5HyGa+F1roXn6kXK1PpIIV49A4fBSI/kGvosq4QFdzzazXRf7jComSERlDIkQmgz5fuox/oBmY1vPX4duO+N/oHJz1/F66xf7zI5XXeBS2zSz0s0hfC4x5kQ4ZEyd3FxK9ahjG9+Fcfg0TyBNVXxNZKTQt3WixefMdn2sLb3WhMXwv5TqJkhEZQyJIJShkQI7TMcB+2ks061p3d/VgDuR9/lUgeHJDtTf/oUjse800zW/3EdOD4ZUVpGHxF8vIQ3HqS9Hel3V/qOX7yFK5KDf6NjbygPXPsN2mfS6mHYGxZqZkgEpQyJENpMzVa24fhelxb6c39dBe7gHXojd/P9s532l07D8cbLZAqmHuNWLW+Z6rS8JD5TieoYHB+cqTI5+ft63/G7IxYcZ7Yp7DaeN4Bbejza9z5hoWaGRFDKkAhKGRIhtM/4/vRv4PgH2htM9rbRLo+mKJVg5LEYjUdrElOq3VHyL7sLuOcjNv8Kk90UpioMpwLH7Qr9mZhPRmwt4viJXQqJjeu4Img+oJRHb8wBTtPDFSuomSERlDIkQmgzdcLELGrGpCzqgXDuk06Byfp6/4WXwZ9+CMfFOwtMNm9jxZU/SaGlvlrDGyXRGOVi433HBAjWJTDpWQ0czPbG+HM9sSJCman/eyhlSASlDIkQWRFbo0ephITAPeRCRL7DjQh+S5mmaZq5Q4v+/kwVuO4gZUqtNXymvGMY2ppNyuoK5W2aMU/bnRN7yLkpykybU+h3XIvzC7Fo6m7VzJAIShkSQSlDIoT2GV6AdrLjUCrBElLYgxa9edwbxuuMU9RVR2wXwafJ49NYcWG16D3H28UVOqOBbzre/Yef/QP+hfZUgcmZZ5jeN/cozeFUsBtQYHB/h1h5fdhy5iFQM0MiKGVIhNBmasfHggRDp6DRrWJzxwGDCr6cMoav9XOUjS0IdQQ83FXslKPp3Pg+mpfDzJIIvsFX5gmmeLpDFD47GSF8ztDfG3OFvRuH1fMeAjUzJIJShkRQypAIoX3Gko1tHwwuJWAXMYV9bY22Jb80j5Uje7Nkl73aOeDaFUqsNOYEm52m8YyOsK8jiaHl7I+5kLlUAC7gXE+vLLTLsHROFsJXLrQNItqFrGaGRFDKkAihzdTtDmZRk3EKWRvjQp/Zj8jcrBexsKBj07mDaQyX9yboWekNY/hqtImzi5iL9VNCbpYzTXa1AFS3SGPuTaIpGrlKb+C9/CH/qoha7KiZIRGUMiSCUoZECO0ztmxsvJhPUCqhNo5GdPwvxK18Dq+b+Tn3Y97H6pDJG5T9DfJZ4IKHVC0SLBwHTu9gURmfHjGE7hT5i9SgeOs8+rp4p38zS351L+bhM63SIUcAShkSIbSZ2rZxm26RM1N2AUNLx6Jh4kk0Iam7m0wWNwXXvjbFZGsT75kZJrO1dRa3cflCRUTx+Hkmp9+9Cpx54wGTY2/N44UemaJeEV+zY3HuDdxVBQlHDkoZEkEpQyKE9hn1LvqMmRLt8fOFLmVBnELdU9VnwNlpKn4zcrjon2xymeABDJd7JynD6mDU+xmfYWfpmRPLrr0WFTMkNzEdY24QZxcwoxvwxc5xsTQuHNTMkAhKGRIhtJna3EPbUB3ltuYKKj4Yoen/WgH3Unzk0ucV3BbWP5U/oXvWF/HrX3wTr/IymsXWJJqb3Vn6QdjnQIPChsJ9oR+uTWF4IGaCHe6PNJWZOnJQypAIShkSIbTPOGgLi/f8yr4Q6rVmSPaEtml+QYhLOei7VDOb3sGw103TfRItYb+dUGAHnIlxrzFC3/1INdD3OFValYz18HfzhWu+8hlHD0oZEiH8loAuXsqbKTOFZsNPUBz6q5WzwFUqlHFNLZwEzi70b6rlc8N3S8KfIRQIxLkWtLxZ0jRN87lFK91Gc9Oc47rsBELY60ZUhcBBzQyJoJQhEZQyJEL4rcfC4lbbo1A3m8EGju19yvCaH2CImq5RB54gLizsc8cxYTXNyZDNFjuxpepo39PbnA9zcKUx9mSDflsatxc7Gfqb4vtCra8VUYEtBzUzJIJShkSI7mtkTSr7nytvAVe/QLHlyn1stGhtUDa2eH0HuPhl2ldm2Ng0zH/zZSb3BvCNu3R1E473Tg+R/Arums2uUFNIXzCTusNtOxA+8+OU1Gd+jjSUMiSCUoZEiMxn3NmgNMOZsafAfXX0Eyanqhha/vKFzzN57b0p4I79luuE9il+STmzSi1wEkUsMxC7GdQucWFoAsPe0hXKzBo9oDQ7T37Czomfm4smU8tDzQyJoJQhESIzU709eltd2hjpe54vND1JcF/Xnbv0ALibQ9NMHn/vPHD7VQpn09vCR3iFr5qlntMzl8ToWXM5C9fBhK7WG6T7+hkhlP0vfJ1dzQyJoJQhEZQyJMI/ATU1xV0gaRmWAAAAAElFTkSuQmCC\" id=\"image8be0fe8c73\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"689.433962\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_43\">\n",
       "    <path d=\"M 689.433962 78.266038 \n",
       "L 689.433962 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_44\">\n",
       "    <path d=\"M 760.5 78.266038 \n",
       "L 760.5 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_45\">\n",
       "    <path d=\"M 689.433962 78.266038 \n",
       "L 760.5 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_46\">\n",
       "    <path d=\"M 689.433962 7.2 \n",
       "L 760.5 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pb5a419526e\">\n",
       "   <rect x=\"7.2\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"pa07450cf78\">\n",
       "   <rect x=\"92.479245\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"p5c008fc8f5\">\n",
       "   <rect x=\"177.758491\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"p63edc89eaf\">\n",
       "   <rect x=\"263.037736\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"pf07da0e887\">\n",
       "   <rect x=\"348.316981\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"p5b63fbe3ef\">\n",
       "   <rect x=\"433.596226\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"pbad3f2c649\">\n",
       "   <rect x=\"518.875472\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"p31735946a1\">\n",
       "   <rect x=\"604.154717\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"pb3c07608f2\">\n",
       "   <rect x=\"689.433962\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 1350x150 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"767.7pt\" height=\"85.466038pt\" viewBox=\"0 0 767.7 85.466038\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n",
       " <metadata>\n",
       "  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n",
       "   <cc:Work>\n",
       "    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n",
       "    <dc:date>2024-03-01T02:17:43.249725</dc:date>\n",
       "    <dc:format>image/svg+xml</dc:format>\n",
       "    <dc:creator>\n",
       "     <cc:Agent>\n",
       "      <dc:title>Matplotlib v3.7.2, https://matplotlib.org/</dc:title>\n",
       "     </cc:Agent>\n",
       "    </dc:creator>\n",
       "   </cc:Work>\n",
       "  </rdf:RDF>\n",
       " </metadata>\n",
       " <defs>\n",
       "  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 85.466038 \n",
       "L 767.7 85.466038 \n",
       "L 767.7 0 \n",
       "L 0 0 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 7.2 78.266038 \n",
       "L 78.266038 78.266038 \n",
       "L 78.266038 7.2 \n",
       "L 7.2 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#pefd602ca44)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAACYElEQVR4nO3dO24TURxGcc/4EcsGIyU8BAIKGpBSRECRQEWBKKkoEZTsgUXQsACWgGgpKKFKQREJFNGFR0NAOFixjT2UlPMVI3IinV/91x1Hx7e4mpm4uFs8qFpCKI/6A+gfY4AYA8QYIMYAMQaIMUCMAWIMEGOAGAPEGCDGADEGiDFAjAFiDBBjgBgDpLP7fDMa7P3Iug2+ZhderBTR3LJbPzO5sIzWqtZm0Vx/tx/NXXwzieZmo+CPaLkzUIwBYgwQY4AYA8QYIMYAMQaIMUA6929vR4N3Rh+iuVffr0dzw3Z2Gn649rZ2ZqvfjtZ6sncrmnv/eiOaKyfzaG5/a5CtF03pvzAGiDFAjAFiDBBjgBgDxBggxgAprj19Fr0Hvrwxjhac/u5Fc9Vhdmo+ce6gdqbbXkRrtV+uRnOrL95Fc01zZ4AYA8QYIMYAMQaIMUCMAWIMEGOAFOl/YiturkcLzk9lT3B3DrJ74NMz9evNTman+cG37Jqd8TSaa5XZd3nZyz6fOwPEGCDGADEGiDFAjAFiDBBjgHTSwWp7p9kFQ/1u/W3c4dnT0Vq/Ni9Fc58eZ69Fb1zZi+b2D33w+dgxBogxQIwBYgwQY4AYA8QYIMYAafrA3LhqXn+r9M/nL9FavZ/nw4tmJ/BUWWS/MebOADEGiDFAjAFiDBBjgBgDxBggxgAp7g0fNfsT1EV4ek0fGh5nrzw3qb1+NZqbXB5Fc+XCE/ixYwwQY4AYA8QYIMYAMQaIMUCMAYK/B34UFjsfo7mV7MH8mDsDxBggxgAxBogxQIwBYgwQY4AYA+Qv7+5Slq9CoVQAAAAASUVORK5CYII=\" id=\"image6679475857\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"7.2\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 7.2 78.266038 \n",
       "L 7.2 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 78.266038 78.266038 \n",
       "L 78.266038 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 7.2 78.266038 \n",
       "L 78.266038 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 7.2 7.2 \n",
       "L 78.266038 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_2\">\n",
       "   <g id=\"patch_7\">\n",
       "    <path d=\"M 92.479245 78.266038 \n",
       "L 163.545283 78.266038 \n",
       "L 163.545283 7.2 \n",
       "L 92.479245 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#pd51e22512a)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAACVElEQVR4nO3dvWpTcQCG8ZwmNbFGatFBUr8Gi4MiVhShg1oQvAFXHZy8ABdvwIsQL8A76CSCUroWg6ho/YCCFCJpxVYazzGOjucdCn2Q5ze/HFIe/sOfQ9LiVnFn3BDCxH5/AP1jDBBjgBgDxBggxgAxBogxQIwBYgwQY4AYA8QYIMYAMQaIMUCMAWIMEGOAGAPEGCDGADEGiDFAjAFiDBBjgBgDxBggrfVHC9HwWL+Mdt03G9Gu/Pw12v0PWqdPRjtPBogxQIwBYgwQY4AYA8QYIMYAMQZIUX07G30PfHV3N3rg08H1aLe0PB/tei/rP97hd8PoWX/Wslv/OPxbWydmo93HB6einScDxBggxgAxBogxQIwBYgwQY4AYA6R4uJr9Etu9mZXogb1m9sNun8pWtHs2vFa7efvjePSsD8tnot3si1G0K7vNaPf97na082SAGAPEGCDGADEGiDFAjAFiDJBicfFxdEvbuNKJHvjzfPbK8urcl2i3MLNWu7k59T561pPBjWi31L8Q7drrk9Hudze7CHsyQIwBYgwQY4AYA8QYIMYAMQaIMUCKvf4HWBOd7KZeXT4X7QYXp2o3w/nsa9EHj/6KdqNR9kq43a//bI1Go9F7tRPtPBkgxgAxBogxQIwBYgwQY4AYA8QYIHt+A98PxeSBbHgpu/VvzR2Kdu3NKtp1nr+Odp4MEGOAGAPEGCDGADEGiDFAjAFiDJDi9vT96AY+rrLbZvrDWeMye29N1jwyHe2qza1o58kAMQaIMUCMAWIMEGOAGAPEGCDGAPkLkplfnrIbQ+IAAAAASUVORK5CYII=\" id=\"image20528f5f22\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"92.479245\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_8\">\n",
       "    <path d=\"M 92.479245 78.266038 \n",
       "L 92.479245 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_9\">\n",
       "    <path d=\"M 163.545283 78.266038 \n",
       "L 163.545283 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_10\">\n",
       "    <path d=\"M 92.479245 78.266038 \n",
       "L 163.545283 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_11\">\n",
       "    <path d=\"M 92.479245 7.2 \n",
       "L 163.545283 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_3\">\n",
       "   <g id=\"patch_12\">\n",
       "    <path d=\"M 177.758491 78.266038 \n",
       "L 248.824528 78.266038 \n",
       "L 248.824528 7.2 \n",
       "L 177.758491 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#paf9077119b)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAACTElEQVR4nO3czUpUcQCG8XNmnCnGjzQDTQwilMwkWhQUtaxAaFXRNmjhHbTxAlx6BbYJr8AiaBvtCoJQgpjogwqkbDGhkmm2bHnexYDP4vmtXw4jD//FH46nvFbe2S+EUDvoH6D/jAFiDBBjgBgDxBggxgAxBogxQIwBYgwQY4AYA8QYIMYAMQaIMUCMAWIMEGOAGAPEGCDGADEGiDFAjAFiDBBjgBgDpOfdwwvR8MT4RrQb7e1Eu1ftk9Fu5FmjcnP0+efoWbtfv0W7g+LJADEGiDFAjAFiDBBjgBgDxBggxgApF1Zno/8Dv9n/JnrgVONQtNst9qLdwo/zlZtHLy9HzxpYa0a74bWdaNfoZLtPs33RzpMBYgwQY4AYA8QYIMYAMQaIMUCMAVJOP1iMbuC/h7MPtvVM/op2c2deRLvrvW8rN7Ui+21LG1ej3dOVS9HucPZaQHH89sdo58kAMQaIMUCMAWIMEGOAGAPEGCBltz+/XR88Eu2+35qOds2765WbJ2eXo2cN1VvRbn79XLRb7YxFu5mB7IVrTwaIMUCMAWIMEGOAGAPEGCDGADEGSNdv4N1WnzxVuWnfH4meNXOlHe2GmtvR7svmYLR7/3o82nkyQIwBYgwQY4AYA8QYIMYAMQaIMUDKG333ohv4362t7In73At9/fREtPt58Vi0+9Mqo93o4w/RzpMBYgwQY4AYA8QYIMYAMQaIMUCMARLfwIta2G0v+6hXfKMHq7Wyt9rTv9WTAWIMEGOAGAPEGCDGADEGiDFAjAHyD8ASUzjSvPIxAAAAAElFTkSuQmCC\" id=\"imagea5e137b50e\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"177.758491\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_13\">\n",
       "    <path d=\"M 177.758491 78.266038 \n",
       "L 177.758491 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_14\">\n",
       "    <path d=\"M 248.824528 78.266038 \n",
       "L 248.824528 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_15\">\n",
       "    <path d=\"M 177.758491 78.266038 \n",
       "L 248.824528 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_16\">\n",
       "    <path d=\"M 177.758491 7.2 \n",
       "L 248.824528 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_4\">\n",
       "   <g id=\"patch_17\">\n",
       "    <path d=\"M 263.037736 78.266038 \n",
       "L 334.103774 78.266038 \n",
       "L 334.103774 7.2 \n",
       "L 263.037736 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#p5e170cc2bc)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAACTUlEQVR4nO3dsUuUYQDH8Xu7i1sOTPQIt2xJKIIaKoWiof6DdBaamhuLplqiaIvGlpbEP6BBuElxKJdqEBFBUkHERSji9BpbgvcXPHFf4vuZf9x78uUZHjzP6k51b9AIHM1eT2aNvZkq2p2M9KPd+Xf1b6+19DF6LbpTw34D+s0YIMYAMQaIMUCMAWIMEGOAGAOklQ7PLG9Hu/bhRLQ7uNiOdq2l5WiX2Ho6He3OPV4p9sy/4ckAMQaIMUCMAWIMEGOAGAPEGCDGAIlv4P1vO9Hu9M5utBtczm7DJT2ZfR/tenenot3XF5eiXWdhNdp5MkCMAWIMEGOAGAPEGCDGADEGiDFA4ht4bBB9qL1RnRR/cq2Xr+ai3Y+x7JP07W723E4282SQGAPEGCDGADEGiDFAjAFiDBBjgBS/gTfHx6Ld9xtHpR9dq/tmOJ8uT3kyQIwBYgwQY4AYA8QYIMYAMQaIMUCK38B35y5Eu/Vbr6PdtfkHtZvRt9nN+vj21WjX7H2KdqV5MkCMAWIMEGOAGAPEGCDGADEGSPFL3+jGz2g3+eF+tJta/Fy76d+8Er3W4cPsV73jvWhWnCcDxBggxgAxBogxQIwBYgwQY4AYA6RK/wHW/6DZzf5W+Hh//x+/kz/zZIAYA8QYIMYAMQaIMUCMAWIMEGOAlP/yL7Bh3axTngwQY4AYA8QYIMYAMQaIMUCMAWIMkNbWs/DfMj/ifnHW5vPsZzi7mn3nd2cz+7T6YO1LtEt5MkCMAWIMEGOAGAPEGCDGADEGiDFAfgFAtExLa0SOwwAAAABJRU5ErkJggg==\" id=\"imagea922c6e07e\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"263.037736\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_18\">\n",
       "    <path d=\"M 263.037736 78.266038 \n",
       "L 263.037736 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_19\">\n",
       "    <path d=\"M 334.103774 78.266038 \n",
       "L 334.103774 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_20\">\n",
       "    <path d=\"M 263.037736 78.266038 \n",
       "L 334.103774 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_21\">\n",
       "    <path d=\"M 263.037736 7.2 \n",
       "L 334.103774 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_5\">\n",
       "   <g id=\"patch_22\">\n",
       "    <path d=\"M 348.316981 78.266038 \n",
       "L 419.383019 78.266038 \n",
       "L 419.383019 7.2 \n",
       "L 348.316981 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#p4af294a76a)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAACgUlEQVR4nO3dz0pUYRyH8ffMvOWguQknkIQywoIipkW7/mzctG/XBXQB3VZdgitdBYIUSlEUgWJGElk0Y+g5rSXh/GreoWfxfNZfzjny8C6Go2O1XD1qUkB973Zklt49PhPa9T7l0O78Vt26yT/bNyml9HUpds/5te+hXXrxKrYL6hS9msZiDBBjgBgDxBggxgAxBogxQIwBUkU/gZfWuXU9tKtfvi52z/ruILTrrG0Uu+ff8GSAGAPEGCDGADEGiDFAjAFiDBBjgMReCk9AyU/WqapCs3wwCu1ib9TL82SAGAPEGCDGADEGiDFAjAFiDBBjgPy3T+BRnV6vfdOfC12rGf4a93FOyIuXQrujDx9DO08GiDFAjAFiDBBjgBgDxBggxgAxBkju3rgWGh5vvpnwo5yuHiy1bqr9H6FrHb99P+7jnNDkbtHreTJAjAFiDBBjgBgDxBggxgAxBogxQPJwYTY0nP52MbQ72t4Z53n+0BkdtW6qUdl321GlP9F7MkCMAWIMEGOAGAPEGCDGADEGSO6tboWGzey5CT/K6eqN9uer+v2i96ympkK75vCw6H09GSDGADEGiDFAjAFiDBBjgBgDxBgguR4OQ8Pu5YXYFfc+j/E4/6gp/HVdN6/GduubRW/ryQAxBogxQIwBYgwQY4AYA8QYIMYAyd0LsffH2w9jX7DVn4/9InVeWQ/tIpph7Gu1o39mfbAYe98/U+5HSCl5MlCMAWIMEGOAGAPEGCDGADEGiDFAchX851Ezy3uh3crT56HdlWdPQrv51fbN7v3QpdKDO7HfuP+yH3unPr07CO1Gc2dDO08GiDFAjAFiDBBjgBgDxBggxgAxBshv3C1d3enUC98AAAAASUVORK5CYII=\" id=\"imagedd54aabd86\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"348.316981\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_23\">\n",
       "    <path d=\"M 348.316981 78.266038 \n",
       "L 348.316981 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_24\">\n",
       "    <path d=\"M 419.383019 78.266038 \n",
       "L 419.383019 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_25\">\n",
       "    <path d=\"M 348.316981 78.266038 \n",
       "L 419.383019 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_26\">\n",
       "    <path d=\"M 348.316981 7.2 \n",
       "L 419.383019 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_6\">\n",
       "   <g id=\"patch_27\">\n",
       "    <path d=\"M 433.596226 78.266038 \n",
       "L 504.662264 78.266038 \n",
       "L 504.662264 7.2 \n",
       "L 433.596226 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#p1e0ab1d858)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAACoUlEQVR4nO3cP2tTYQCF8ffe3CaKEpSmrVRFW7EUOthCURDUwVYRFBwciojg4uIHUOj3cJNm6OKgH0IEY6GCSyvaImKgQypo09RE88fRRXjPEPUMz28+vEl4uMMll5vMJbd6ARbS//0F8BsxjBDDCDGMEMMIMYwQwwgxjBDDCDGMEMMIMYwQwwgxjBDDCDGMEMMIMYwQw0h24MWQNGx2BqTd+uaotBusaOeV3u5GN7kvdems0Pohzbpfv0m7Xqsl7dLxE9pOWuGfIIYRYhghhhFiGCGGEWIYIYYRYhjJLpfeScMHhz5Lu+qp+B1zCCGUz85KuycvL0Y348/y0ln5ivZbu42GtFMlje/SjivDCDGMEMMIMYwQwwgxjBDDCDGMEMNI9nj5hjR8emlL2t0+viLt7h9elXYTc/HPfdRZkM6a3Doq7cLae22nymv/93NlGCGGEWIYIYYRYhghhhFiGCGGEWIYSdQ3seVGhqUD6+fHpF31ekfa3Z19Fd2s149IZ20sT0i74aU30k59Cj07pt35c2UYIYYRYhghhhFiGCGGEWIYIYYR+aav79KcNKs+PBfdjF39KJ21WStJu5HyPmlXqDWl3fb0QWnHlWGEGEaIYYQYRohhhBhGiGGEGEaIYSSZH1iQ7sB77fbf/i5/lCsWo5sPi1PSWTfnK9Lude2ktGu2M2k3M1SVdlwZRohhhBhGiGGEGEaIYYQYRohhhBhGsiSvvTgrKRSkXb9fnNXZ2YluTpe3pbOej05Lu3tn4g9bhxBCIf0p7dZ2tVeSc2UYIYYRYhghhhFiGCGGEWIYIYYRYhhJruy/oz2Fnva3W3dvr6/nSZ95YUbafbqmPYXe0/4CD8UNbceVYYQYRohhhBhGiGGEGEaIYYQYRohh5Bdy/ml+9wPTtQAAAABJRU5ErkJggg==\" id=\"image5fa0feea7d\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"433.596226\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_28\">\n",
       "    <path d=\"M 433.596226 78.266038 \n",
       "L 433.596226 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_29\">\n",
       "    <path d=\"M 504.662264 78.266038 \n",
       "L 504.662264 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_30\">\n",
       "    <path d=\"M 433.596226 78.266038 \n",
       "L 504.662264 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_31\">\n",
       "    <path d=\"M 433.596226 7.2 \n",
       "L 504.662264 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_7\">\n",
       "   <g id=\"patch_32\">\n",
       "    <path d=\"M 518.875472 78.266038 \n",
       "L 589.941509 78.266038 \n",
       "L 589.941509 7.2 \n",
       "L 518.875472 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#p2dc10664de)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAACZklEQVR4nO3csWpTYQCG4fOfJI1parXYIhV0EMTBxeLUycXFRSgU3HRx8BocvAOvwdVRcHEqCHYSlLpZwWIihYJg1FoQkhNHx3yCtO/wPvPHn8CbMxxyknKrbE4rIdQn/Qb0lzFAjAFiDBBjgBgDxBggxgAxBogxQIwBYgwQY4AYA8QYIMYAMQaIMUCMAWIMEGOAGAPEGCDGADEGiDFAjAFiDBBjgLR3n96IhmffdKPdhReDaDcefol2J6G1shLthvevRLu797ainVcGiDFAjAFiDBBjgBgDxBggxgAxBkh5/P5O9Dvw24s70YHPR9kd/bPt9WjXWz2cuWm3muisH1/70a50svMerG1Hu0fLH6KdVwaIMUCMAWIMEGOAGAPEGCDGADEGSLn+8El0B97bOIgO3Lz4Ntq9PLgW7dbP7c3c7P8+E531eng52h1960W7+mc72i1+zD7zXhkgxgAxBogxQIwBYgwQY4AYA6Skf7/d3FyLDvy0kT0g3fQn0W5h+dfMzeEou0nrDrL3tjDI/pH8/NZ+tBvvfY52XhkgxgAxBogxQIwBYgwQY4AYA8QYINn3hlVV1a/eRburO9lXoNNLq9FucvrU7LPq7EHl0hxFu6YTfkab7HXr+flsl72qjoMxQIwBYgwQY4AYA8QYIMYAMQZIfAeemoy+Z8NwV/7T5l90+tlPlCfjcbQrc3PRzisDxBggxgAxBogxQIwBYgwQY4AYA6TdWlrKltPs+96qyZ7gnk6yp9CnwV1uKeE9eB1+9sJd3c2eaq9arey87DQdB2OAGAPEGCDGADEGiDFAjAFiDJA/YYlZKScNGFQAAAAASUVORK5CYII=\" id=\"image0a5d1c23e3\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"518.875472\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_33\">\n",
       "    <path d=\"M 518.875472 78.266038 \n",
       "L 518.875472 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_34\">\n",
       "    <path d=\"M 589.941509 78.266038 \n",
       "L 589.941509 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_35\">\n",
       "    <path d=\"M 518.875472 78.266038 \n",
       "L 589.941509 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_36\">\n",
       "    <path d=\"M 518.875472 7.2 \n",
       "L 589.941509 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_8\">\n",
       "   <g id=\"patch_37\">\n",
       "    <path d=\"M 604.154717 78.266038 \n",
       "L 675.220755 78.266038 \n",
       "L 675.220755 7.2 \n",
       "L 604.154717 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#pe07e094e33)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAACgUlEQVR4nO3cu2oUYQCG4ZmdyRJJgpIDniAi8YBio4WoWFhItPASBFHxKrwJC8HKznuQ9GIaFQQbBU9shFgEIS64G5MdexH287DsC75P/fEz5OUvJtlseeXU3aYIdI/sTmbFxsk62lVb0aw4cP/50E3T72eHjUnv2tlo1xrxc+g3GAPEGCDGADEGiDFAjAFiDBBjgJTLu65Hb+CDXm/Uz/Lf82aAGAPEGCDGADEGiDFAjAFiDBBjgNStPdnftgfrvoGPmjcDxBggxgAxBogxQIwBYgwQY4AYA6Te2T+fLdc/R7PqxNFo9/rOXLS7dOHV0M3auW501rhUc7PRzpsBYgwQY4AYA8QYIMYAMQaIMUCMAVKXg0E0jD6qXhRF2cv+wbu1XUa7h4tPhm6Wz9+IzipXX0a7aj777UDn5vFot/joXbTzZoAYA8QYIMYAMQaIMUCMAWIMEGOA1E3r3/bYfv8x2h17kJ13+fGtoZtq9UV2WKh7cSkbZr9EKIqqimbeDBBjgBgDxBggxgAxBogxQIwBUhf1eHo0G1+iXRW+REZnLSxEu87V7LyZN3/xML/gzQAxBogxQIwBYgwQY4AYA8QYIMYAqYvwg8/xgYcPZcOt79luc/PPH+Yn385kz1Z9zf5MOv0p+9kNZmeinTcDxBggxgAxBogxQIwBYgwQY4AYA6QuyvTTu5mmPZENJ9vRrG4Pf2tOP2zdXnkW7ZZWoln8pV7l9FS082aAGAPEGCDGADEGiDFAjAFiDBBjgNQ7k3U0rCayN+bBh060a05nX5z19vbwL+KaWjsYnbXv3tNol2p6/Wy4N/uKc28GiDFAjAFiDBBjgBgDxBggxgAxBsgPw4FWiVEgNL8AAAAASUVORK5CYII=\" id=\"imagea82380e501\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"604.154717\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_38\">\n",
       "    <path d=\"M 604.154717 78.266038 \n",
       "L 604.154717 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_39\">\n",
       "    <path d=\"M 675.220755 78.266038 \n",
       "L 675.220755 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_40\">\n",
       "    <path d=\"M 604.154717 78.266038 \n",
       "L 675.220755 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_41\">\n",
       "    <path d=\"M 604.154717 7.2 \n",
       "L 675.220755 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       "  <g id=\"axes_9\">\n",
       "   <g id=\"patch_42\">\n",
       "    <path d=\"M 689.433962 78.266038 \n",
       "L 760.5 78.266038 \n",
       "L 760.5 7.2 \n",
       "L 689.433962 7.2 \n",
       "z\n",
       "\" style=\"fill: #ffffff\"/>\n",
       "   </g>\n",
       "   <g clip-path=\"url(#p052086c483)\">\n",
       "    <image xlink:href=\"data:image/png;base64,\n",
       "iVBORw0KGgoAAAANSUhEUgAAAGMAAABjCAYAAACPO76VAAACaElEQVR4nO3cTWoTcQCG8ZnJQGzVhILaWj9aiyC4EnHlwoUouHHnATyCR/AIrjyK7tSVG0HERSsoIrSiiEQtIaSaSbxAYV5rwAd8fuuXScrT/2I6Q8sb5Z1ZMU9lGc32bl2Jdt3HL9pHVSe6Vr26Eu0mOx+j3bxV/+RTtS9jgBgDxBggxgAxBogxQIwBYgyQcu534DowTwaIMUCMAWIMEGOAGAPEGCDGADEGiDFAjAFiDBBjgBgDxBggxgAxBogxQIwBUtcb69FwNvgW7ZrvP/7i6/zfPBkgxgAxBogxQIwBYgwQY4AYA8QYIPXs6yAaTi+sRbtP145Gu/Hx7OX36mf7pjkUXaoom2w3rbPd6aeTaFePsg/2ZIAYA8QYIMYAMQaIMUCMAWIMEGOA1M3ubjSs3nyIdr2zF6PdkZ3wP7b123eDy9kd7sKxUbT79S77K0L3UfBf4v6AJwPEGCDGADEGiDFAjAFiDBBjgIQPGHO9zewxbrP1Ntr1FxdbN9P6UnSt4Vp2M7e0Fc3mzpMBYgwQY4AYA8QYIMYAMQaIMUCMARLfgVe97O61Odw98JfZz3TU/qj0xMPn0bVWz2Uvbxez7KXs7LXnnCcDxBggxgAxBogxQIwBYgwQY4AYAyS+A29WlqLd9s1etKuuX412p560v5hdbr6PrjXrZL97k+V+tOt8/hLtpuNxtPNkgBgDxBggxgAxBogxQIwBYgwQY4Dkz8CHe9FudCZ7Mvzy9oNot3Sv/S3088/uRtdaeNV+raIoiuFG9jOs31+OdtuvT0Y7TwaIMUCMAWIMEGOAGAPEGCDGADEGyG/U/1ejA+chtgAAAABJRU5ErkJggg==\" id=\"imagef2996910e1\" transform=\"scale(1 -1) translate(0 -71.28)\" x=\"689.433962\" y=\"-6.986038\" width=\"71.28\" height=\"71.28\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_43\">\n",
       "    <path d=\"M 689.433962 78.266038 \n",
       "L 689.433962 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_44\">\n",
       "    <path d=\"M 760.5 78.266038 \n",
       "L 760.5 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_45\">\n",
       "    <path d=\"M 689.433962 78.266038 \n",
       "L 760.5 78.266038 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_46\">\n",
       "    <path d=\"M 689.433962 7.2 \n",
       "L 760.5 7.2 \n",
       "\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pefd602ca44\">\n",
       "   <rect x=\"7.2\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"pd51e22512a\">\n",
       "   <rect x=\"92.479245\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"paf9077119b\">\n",
       "   <rect x=\"177.758491\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"p5e170cc2bc\">\n",
       "   <rect x=\"263.037736\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"p4af294a76a\">\n",
       "   <rect x=\"348.316981\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"p1e0ab1d858\">\n",
       "   <rect x=\"433.596226\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"p2dc10664de\">\n",
       "   <rect x=\"518.875472\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"pe07e094e33\">\n",
       "   <rect x=\"604.154717\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       "  <clipPath id=\"p052086c483\">\n",
       "   <rect x=\"689.433962\" y=\"7.2\" width=\"71.066038\" height=\"71.066038\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<Figure size 1350x150 with 9 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 代码验证\n",
    "# 导入PyTorch和d2l库\n",
    "import torch\n",
    "from torch import nn\n",
    "from d2l import torch as d2l\n",
    "\n",
    "# 定义一个包含多个卷积层和全连接层的LeNet模型\n",
    "# 定义神经网络结构\n",
    "# 创建一个序列化的神经网络模型\n",
    "net = nn.Sequential(\n",
    "    # 第一个二维卷积层。输入通道为1（例如，用于灰度图像），输出通道为6。使用5x5的卷积核，边缘填充2个像素。\n",
    "    nn.Conv2d(1, 10, kernel_size=5, padding=2), \n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # 最大池化层，用于降低特征维度。使用2x2的窗口大小和步长2，减少数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第二个二维卷积层。输入通道为6（上一卷积层的输出通道数），输出通道为16。使用5x5的卷积核。\n",
    "    nn.Conv2d(10, 16, kernel_size=5), \n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # 又一个最大池化层，进一步减小数据的空间尺寸。\n",
    "    nn.MaxPool2d(kernel_size=2, stride=2),\n",
    "    # 第三个二维卷积层。输入通道为16（上一卷积层的输出通道数），输出通道为120。使用5x5的卷积核。\n",
    "    nn.Conv2d(16, 120, kernel_size = 5),\n",
    "    # 使用ReLU激活函数。\n",
    "    nn.ReLU(),\n",
    "    # Flatten层，将多维度的数据展平成一维，以便传递给全连接层。\n",
    "    nn.Flatten(),\n",
    "    # 全连接（线性）层。输入特征数量为16*5*5（从上一层的输出尺寸计算而来），输出特征数量为120。\n",
    "    nn.Linear(120, 120), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第二个全连接（线性）层，进一步转换特征。输入特征数量为120，输出为84。\n",
    "    nn.Linear(120, 84), \n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第三个全连接（线性）层，进一步转换特征。输入特征数量为84，输出为20。\n",
    "    nn.Linear(84, 20),\n",
    "    # 使用Sigmoid激活函数。\n",
    "    nn.Sigmoid(),\n",
    "    # 第四个全连接（线性）层，最终输出特征数量为10，通常用于10类分类问题。\n",
    "    nn.Linear(20, 10))\n",
    "\n",
    "# 创建一个随机输入张量\n",
    "X = torch.rand(size=(1, 1, 28, 28), dtype=torch.float32)\n",
    "\n",
    "# 遍历LeNet模型的各层并输出各层的输出形状\n",
    "for layer in net:\n",
    "    X = layer(X)\n",
    "    print(layer.__class__.__name__,'output shape: \\t',X.shape)\n",
    "\n",
    "# 设置训练批量大小\n",
    "batch_size = 256\n",
    "\n",
    "# 加载Fashion MNIST数据集并创建训练和测试迭代器\n",
    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size=batch_size)\n",
    "\n",
    "# 定义用于在GPU上评估模型精度的函数\n",
    "def evaluate_accuracy_gpu(net, data_iter, device=None): #@save\n",
    "    \"\"\"使用GPU计算模型在数据集上的精度\"\"\"\n",
    "    if isinstance(net, nn.Module):\n",
    "        net.eval()  # 设置为评估模式\n",
    "        if not device:\n",
    "            device = next(iter(net.parameters())).device\n",
    "    # 正确预测的数量，总预测的数量\n",
    "    metric = d2l.Accumulator(2)\n",
    "    with torch.no_grad():\n",
    "        for X, y in data_iter:\n",
    "            if isinstance(X, list):\n",
    "                # BERT微调所需的（之后将介绍）\n",
    "                X = [x.to(device) for x in X]\n",
    "            else:\n",
    "                X = X.to(device)\n",
    "            y = y.to(device)\n",
    "            metric.add(d2l.accuracy(net(X), y), y.numel())\n",
    "    return metric[0] / metric[1]\n",
    "\n",
    "# 定义用于训练模型的函数@save\n",
    "def train_ch6(net, train_iter, test_iter, num_epochs, lr, device):\n",
    "    \"\"\"用GPU训练模型(在第六章定义)\"\"\"\n",
    "    # 定义初始化权重的函数，使用Xavier初始化\n",
    "    def init_weights(m):\n",
    "        if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
    "            nn.init.xavier_uniform_(m.weight)\n",
    "    # 对模型的所有层应用初始化权重的函数\n",
    "    net.apply(init_weights)\n",
    "    # 打印在哪个设备上进行训练（CPU或GPU）\n",
    "    print('training on', device)\n",
    "    # 将模型移动到指定的设备（CPU或GPU）\n",
    "    net.to(device)\n",
    "    # 定义优化器，使用随机梯度下降（SGD）优化器\n",
    "    optimizer = torch.optim.SGD(net.parameters(), lr=lr)\n",
    "    # 定义损失函数，使用交叉熵损失\n",
    "    loss = nn.CrossEntropyLoss()\n",
    "    # 创建用于绘制训练过程的动画\n",
    "    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n",
    "                            legend=['train loss', 'train acc', 'test acc'])\n",
    "    # 创建计时器和获取每个epoch的批量数\n",
    "    timer, num_batches = d2l.Timer(), len(train_iter)\n",
    "    for epoch in range(num_epochs):\n",
    "        # 训练损失之和，训练准确率之和，样本数\n",
    "        metric = d2l.Accumulator(3)\n",
    "        # 设置模型为训练模式\n",
    "        net.train()\n",
    "        for i, (X, y) in enumerate(train_iter):\n",
    "            timer.start()\n",
    "            optimizer.zero_grad()\n",
    "            # 将输入数据和标签移动到指定的设备上\n",
    "            X, y = X.to(device), y.to(device)\n",
    "            # 前向传播\n",
    "            y_hat = net(X)\n",
    "            # 计算损失\n",
    "            l = loss(y_hat, y)\n",
    "            # 反向传播\n",
    "            l.backward()\n",
    "            # 更新模型参数\n",
    "            optimizer.step()\n",
    "            with torch.no_grad():\n",
    "                # 记录训练损失、训练准确率和样本数\n",
    "                metric.add(l * X.shape[0], d2l.accuracy(y_hat, y), X.shape[0])\n",
    "            timer.stop()\n",
    "            train_l = metric[0] / metric[2]\n",
    "            train_acc = metric[1] / metric[2]\n",
    "            # 每完成一定批量数的训练，更新动画\n",
    "            if (i + 1) % (num_batches // 5) == 0 or i == num_batches - 1:\n",
    "                animator.add(epoch + (i + 1) / num_batches,\n",
    "                            (train_l, train_acc, None))\n",
    "        # 计算并记录测试集上的准确率\n",
    "        test_acc = evaluate_accuracy_gpu(net, test_iter)\n",
    "        animator.add(epoch + 1, (None, None, test_acc))\n",
    "\n",
    "    # 以下代码用于可视化模型中的某些层的输出\n",
    "    x_first_Sigmoid_layer = net[0:2](X)[0:9, 1, :, :]\n",
    "    d2l.show_images(x_first_Sigmoid_layer.reshape(9, 28, 28).cpu().detach(), 1, 9)\n",
    "    x_second_Sigmoid_layer = net[0:5](X)[0:9, 1, :, :]\n",
    "    d2l.show_images(x_second_Sigmoid_layer.reshape(9, 10, 10).cpu().detach(), 1, 9)\n",
    "\n",
    "    # 打印最终的训练损失、训练准确率和测试准确率\n",
    "    print(f'loss {train_l:.3f}, train acc {train_acc:.3f}, '\n",
    "        f'test acc {test_acc:.3f}')\n",
    "    # 打印训练速度（样本/秒）\n",
    "    print(f'{metric[2] * num_epochs / timer.sum():.1f} examples/sec '\n",
    "        f'on {str(device)}')\n",
    "\n",
    "# 设置学习率和训练周期\n",
    "lr, num_epochs = 0.1, 50\n",
    "\n",
    "# 调用train_ch6函数来在GPU上训练模型\n",
    "train_ch6(net, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())\n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
